Design Patterns & Concurrency

(1)

Sebastian Graf, Oliver Haase

Design Patterns & Concurrency

(2)

Fundamentals

2

(3)

Recap

‣ Threadsafety

• What is threadsafety?

• How can threadsafety be ensured?

‣ Amdahls Law

(4)

Recap Threadsafety

4 public final class HitCounter {

/** internal state counter variable */

private int counter = 0;

/**

* Getting the counter right now * @return the counts

*/

public final int getCounter() { return this.counter;

} /**

* Counting multiple values

* @param countsToAdd numbers to add to the counter */

public final void increment(final int countsToAdd) { counter = counter + countsToAdd;

}

/** Counting by one */

public final void incrementByOne() { increment(1);

} }

(5)

Recap Threadsafety

public final class SynchronizedCounter { /** internal state counter variable */

private int counter = 0;

/**

* Getting the counter right now * @return the counts

*/

public synchronized final int getCounter() { return this.counter;

} /**

* Counting multiple values

* @param countsToAdd numbers to add to the counter */

public synchronized final void increment(final int countsToAdd) { counter = counter + countsToAdd;

}

/** Counting by one */

public final void incrementByOne() { increment(1);

} }

(6)

Recap Threadsafety

6 import java.util.List;

public final class ListSum { /**

* Summing multiple variables *

* @param multiple

* variables

* @return the sum of the variables */

public final int sum(final List<Integer> factors) { int sum = 0;

for (final int number : factors) { sum = sum + number;

}

return sum;

} }

(7)

Recap Threadsafety

public final class IDHandler { /**Local set of ids*/

private String[] ids = { "e634g", "6fhg4", "hd4fg", "dgf53" };

/**Getting the ids.

*

* @return the ids */

public String[] getIDs() { return ids;

} }

(8)

Recap Threadsafety

8

private final List<String >ids = new ArrayList()<String>;

public IDHandler() { ids.add("e634g");

ids.add("6fhg4");

ids.add("hd4fg");

ids.add("dgf53");

}

/**Getting the ids.

*

public String[] getIDs() {

return ids.toArray(new String[4]);

} }

(9)

Threadsafe?

public class ReadyStater {

private static boolean ready;

private static int number;

private static class ReaderThread extends Thread { public void run() {

while (!ready)

Thread.yield();

System.out.println(number);

} }

public static void main(String[] args) { new ReaderThread().start();

number = 42;

ready = true;

}

(10)

Locking & Visibility

10

(11)

Difference regarding visibility

@NotThreadSafe

public class MutableInteger { private int value;

public int get() { return value; }

public void set(int value) { this.value = value; } }

@NotThreadSafe

public class MutableInteger { private long value;

public long get() { return value; }

public void set(long value) { this.value = value; }

}

(12)

Always Synchronize?

12

public class MutableInteger {

private volatile long value;

public long get() { return value; }

public void set(long value) { this.value = value; } }

public class MutableInteger { private long value;

public synchronized long get() { return value; }

public synchronized void set(long value) { this.value = value; }

}

(13)

Volatile VS Synchronize

Volatile

‣ ^{No Locking}

• no synchronization

‣ ^visibility

‣ ^Usage:

• using if only one thread updates value OR

• value is not participating to invariants OR

• any other reason

Synchonize

‣ ^Locking

• synchronization

‣ ^visibility

‣ ^Usage:

• guarding multiple variables (invariants)

• accessing within multiple

threads

(14)

Publishing and Escaping

‣ Publishing: Making a variable visible out of its current scope

‣ Escaping: Doing the same by accident

14

private String[] ids = { "e634g", "6fhg4", "hd4fg", "dgf53" };

/**Getting the ids.

*

public String[] getIDs() { return ids;

} }

(15)

Escaping in Construtor

public class ThisEscape {

public ThisEscape(EventSource source) { source.registerListener(

new EventListener() {

public void onEvent(Event e) { doSomething(e);

} });

}

(16)

Safe Construction Process

Do not allow to escape this during construction.

16

public class SafeListener {

private final EventListener listener;

private SafeListener() {

listener = new EventListener() { public void onEvent(Event e) { doSomething(e);

} };

}

public static SafeListener newInstance(EventSource source) { SafeListener safe = new SafeListener();

source.registerListener(safe.listener);

return safe;

} }

(17)

Further techniques to ensure thread safety

‣ Confinement

‣ Immutability

‣ Publication

‣ ^Design

‣ Documentation

(18)

Confinement

‣ Thread Confinement

Binding an object to exactly one thread (which cannot be enforced by the JVM)

‣ Ad-hoc Confinement

Responsibility of maintaining thread safety is on implementation side (which is mostly fragile)

‣ Stack Confinement

Object bound to local variables only (happening in methods)

‣ ThreadLocal

18

private static ThreadLocal<Connection> connectionHolder = new ThreadLocal<Connection>() {

public Connection initialValue() {

return DriverManager.getConnection(DB_URL);

} };

public static Connection getConnection() { return connectionHolder.get();

}

(19)

Immutability

Criterias for Immutability

‣ State cannot be modified after construction

‣ All fields are final

‣ properly constructed

Immutable Objects are always thread-safe!

public final class ThreeStooges {

private final Set<String> stooges = new HashSet<String>();

public ThreeStooges() { stooges.add("Moe");

stooges.add("Larry");

stooges.add("Curly");

}

public boolean isStooge(String name) { return stooges.contains(name);

}

(20)

Immutability and Publishing

20

@Immutable

class OneValueCache {

private final BigInteger lastNumber;

private final BigInteger[] lastFactors;

public OneValueCache(BigInteger i,

BigInteger[] factors) { lastNumber = i;

lastFactors = Arrays.copyOf(factors, factors.length);

}

public BigInteger[] getFactors(BigInteger i) {

if (lastNumber == null || !lastNumber.equals(i)) return null;

else

return Arrays.copyOf(lastFactors, lastFactors.length);

} }

@ThreadSafe

public class VolatileCachedFactorizer implements Servlet { private volatile OneValueCache cache =

new OneValueCache(null, null);

public void service(ServletRequest req, ServletResponse resp) { BigInteger i = extractFromRequest(req);

BigInteger[] factors = cache.getFactors(i);

if (factors == null) { factors = factor(i);

cache = new OneValueCache(i, factors);

}

encodeIntoResponse(resp, factors);

} }

(21)

Publishing of data

public Holder holder;

public void initialize() { holder = new Holder(42);

}

public class Holder { private int n;

public Holder(int n) { this.n = n; } public void assertSanity() {

if (n != n)

throw new AssertionError("This statement is false.");

} }

(22)

Safe publication idioms

‣ Initialize objects references from a static initializer

‣ Storing a reference into a volatile or AtomicReference

‣ Storing a reference into a final field of a proper constructed object

‣ Guard the field that stores the object with a lock

22

Publication requirements

‣ Immutable objects: can be published with any mechanism

‣ Effectively immutable objects: must be published safely

‣ Mutable objects: must be published safely and be guared by

a lock

(23)

Useful policies for sharing objects

‣ Thread-confined: Object is owned exclusively by one thread

‣ Shared read-only: Object can be accessed concurrently by multiple threads without additional synchronization

‣ Shared thread-safe: Synchronization on object is handled internally.

‣ Guarded: Access to the object only possible when given

lock held including encluding other thread-safe objects

(24)

Designing Objects

Identify the variables that form the object's state;

• Identify the invariants that constrain the state variables;

• Establish a policy for managing concurrent access to the object's state

24

(25)

States?

State of an object includes state of all fields

‣ primitive: simple

‣ n-primitives: n-tuple of the fields

‣ LinkedList: all linked nodes

Stay aware of pre-/postconditions and invariants:

Synchronization-Policy

(26)

Thread-Safety redefined

Making a class thread-safe means ensuring that its invariants hold under concurrent access; this requires reasoning about its state

26

@ThreadSafe

public final class Counter {

@GuardedBy("this") private long value = 0;

public synchronized long getValue() { return value;

}

public synchronized long increment() { if (value == Long.MAX_VALUE)

throw new IllegalStateException("counter overflow");

return ++value;

} }

(27)

Instance Confinement

Guarding the access with wrapper

‣ Useful for debugging issues

‣ Class itself is thread-safe

(as are the states only accessible within this class)

‣ ^Example: Collections.synchronizedList representing Decorator-Pattern for guarding collections

Caution: wrapped object must not escape!

(28)

Monitor-Based Locking

Each object holds its own lock

28

@ThreadSafe

public final class Counter {

@GuardedBy("this") private long value = 0;

public synchronized long getValue() { return value;

}

public synchronized long increment() { if (value == Long.MAX_VALUE)

throw new IllegalStateException("counter overflow");

return ++value;

} }

(29)

Public VS Private Locking

public class PrivateLock {

private final Object myLock = new Object();

@GuardedBy("myLock") Widget widget;

void someMethod() {

synchronized(myLock) {

// Access or modify the state of widget }

} }

public class PublicLock {

@GuardedBy("this") Widget widget;

void synchronized someMethod() {

// Access or modify the state of widget }

}

(30)

Delegation of Thread safety

30

public class MutablePoint { public int x, y;

public MutablePoint() { x = 0; y = 0; } public MutablePoint(MutablePoint p) { this.x = p.x;

this.y = p.y;

}

(31)

Delegation of Thread safety

@ThreadSafe

public class MonitorVehicleTracker { @GuardedBy("this")

private final Map<String, MutablePoint> locations;

public MonitorVehicleTracker(

Map<String, MutablePoint> locations) { this.locations = deepCopy(locations);

}

public synchronized Map<String, MutablePoint> getLocations() { return deepCopy(locations);

}

public synchronized MutablePoint getLocation(String id) { MutablePoint loc = locations.get(id);

return loc == null ? null : new MutablePoint(loc);

}

public synchronized void setLocation(String id, int x, int y) { MutablePoint loc = locations.get(id);

if (loc == null)

throw new IllegalArgumentException("No such ID: " + id);

loc.x = x;

loc.y = y;

}

private static Map<String, MutablePoint> deepCopy(

Map<String, MutablePoint> m) { Map<String, MutablePoint> result =

new HashMap<String, MutablePoint>();

for (String id : m.keySet())

result.put(id, new MutablePoint(m.get(id)));

return Collections.unmodifiableMap(result);

}

(32)

Better…

32

@Immutable

public class Point {

public final int x, y;

public Point(int x, int y) { this.x = x;

this.y = y;

}

(33)

Better

@ThreadSafe

public class DelegatingVehicleTracker {

private final ConcurrentMap<String, Point> locations;

private final Map<String, Point> unmodifiableMap;

public DelegatingVehicleTracker(Map<String, Point> points) { locations = new ConcurrentHashMap<String, Point>(points);

unmodifiableMap = Collections.unmodifiableMap(locations);

}

public Map<String, Point> getLocations() { return unmodifiableMap;

}

public Point getLocation(String id) { return locations.get(id);

}

public void setLocation(String id, int x, int y) {

if (locations.replace(id, new Point(x, y)) == null) throw new IllegalArgumentException(

"invalid vehicle name: " + id);

} }

(34)

Independence of state variables

34

public class VisualComponent {

private final List<KeyListener> keyListeners = new CopyOnWriteArrayList<KeyListener>();

private final List<MouseListener> mouseListeners = new CopyOnWriteArrayList<MouseListener>();

public void addKeyListener(KeyListener listener) { keyListeners.add(listener);

}

public void addMouseListener(MouseListener listener) { mouseListeners.add(listener);

}

public void removeKeyListener(KeyListener listener) { keyListeners.remove(listener);

}

public void removeMouseListener(MouseListener listener) {

mouseListeners.remove(listener);

} }

public class NumberRange {

// INVARIANT: lower <= upper

private final AtomicInteger lower = new AtomicInteger (0); private final AtomicInteger upper = new AtomicInteger (0);

public void setLower(int i) { if (i > upper.get())

throw new IllegalArgumentException(

"can't set lower to " + i + " > upper");

lower.set(i);

}

public void setUpper(int i) { if (i < lower.get())

throw new IllegalArgumentException(

"can't set upper to " + i + " < lower");

upper.set(i);

}

public boolean isInRange(int i) {

return (i >= lower.get() && i <= upper.get());

} }

(35)

Extending existing sources

‣ Extending? Inheritence? Composition?

‣ Locking is based on functionality to be offered

Alien-Calls in foreign source code complicate things further.

(36)

What to lock?

36 public class ListHelper<E> {

public List<E> list =

Collections.synchronizedList(new ArrayList<E>());

...

public synchronized boolean putIfAbsent(E x) { boolean absent = !list.contains(x);

if (absent)

list.add(x);

return absent;

} }

public class ListHelper<E> { public List<E> list =

Collections.synchronizedList(new ArrayList<E>());

...

public boolean putIfAbsent(E x) { synchronized (list) {

boolean absent = !list.contains(x);

if (absent)

list.add(x);

return absent;

} }

}

(37)

or…Composition…

@ThreadSafe

public class ImprovedList<T> implements List<T> { private final List<T> list;

public ImprovedList(List<T> list) { this.list = list; } public synchronized boolean putIfAbsent(T x) {

boolean contains = list.contains(x);

if (contains)

list.add(x);

return !contains;

}

public synchronized void clear() { list.clear(); } // ... similarly delegate other List methods

}

(38)

Further Example: BlockingQueues

38

Void call()

Producer

BlockingQueue

Void call()

Consumer

Item Item Item Item Item Item Item Item Pill

(39)

Documentation

‣ Each use of volatile / synchronize reflects a synchronization strategy for mainting invariants/pre-/post-conditions

• JavaDoc (often not really usable regarding thread safety)

• Assumptions

• Annotations

(40)

Annotations

‣ Class-Intended:

•

^@Immutable

•

@ThreadSafe

•

@NotThreadSafe

‣ Field- and Method-Intended:

•

@GuardedBy(lock)

•

@GuardedBy(“this“)

•

@GuardedBy(“(Class.)fieldName“)

•

@GuardedBy(“methodName()“)

40

(41)

Outcome of first part

It's the mutable state, stupid.

• Make fields final unless they need to be mutable.

• Immutable objects are automatically thread-safe.

• Encapsulation makes it practical to manage the complexity.

• Guard each mutable variable with a lock.

• Guard all variables in an invariant with the same lock.

• Hold locks for the duration of compound actions.

• A program that accesses a mutable variable from multiple threads without synchronization is a broken program.

• Don't rely on clever reasoning about why you don't need to synchronize.

• Include thread safety in the design processor explicitly document that your class is not thread- safe.

• Document your synchronization policy.

(42)

Structuring Applications

42

(43)

Task Execution

class ThreadPerTaskWebServer {

public static void main(String[] args) throws IOException { ServerSocket socket = new ServerSocket(80);

while (true) {

final Socket connection = socket.accept();

Runnable task = new Runnable() { public void run() {

handleRequest(connection);

} };

new Thread(task).start();

} }

}

(44)

How to start a thread

‣ Starting of threads should be based on defined pool of

threads

‣ Support of lifecycle as well

44

run()

void sum(final int[][] numbers)

run()

Runnable 1 Runnable 2

Thread 0

run()

Runnable 3

run()

Runnable 4

run()

Runnable 5

run()

Runnable 6

(45)

A summing example

Integer call()

int[] sum(final int[][] numbers)

Integer call()

Callable 2 Thread 0

Callable 1

ExecutorService.submit…

FutureList.add…

FutureList.get…

Future.get…

public void sum(final int[][] numbers) {

final List<Future<Integer>> futures = new ArrayList<Future<Integer>>();

final ExecutorService exec = Executors.newCachedThreadPool();

for (int i = 0; i < numbers.length; i++) { final int index = i;

Callable<Integer> summer = new Callable<Integer>() { public Integer call() throws Exception {

return sum;

} };

futures.add(exec.submit(summer));

// getting all futures

for (int i = 0; i < futures.size(); i++) { try {

returnVal[i] = futures.get(i).get();

} catch (final InterruptedException e) { e.printStackTrace();

returnVal[i] = Integer.MIN_VALUE;

} catch (final ExecutionException e) { e.printStackTrace();

returnVal[i] = Integer.MIN_VALUE;

} } }

(46)

Advantages of bound execution

‣ Defined Thread lifecycle

‣ Adaptive and independent resource management

‣ ^Stability

46

(47)

Hazards while not using Executors

run()

void sum(final int[][] numbers)

run()

Runnable 1 Runnable 2

Thread 0

run()

Runnable 3

run()

Runnable 4

run()

Runnable 5

run()

Runnable 6

(48)

Further Hazards

‣ Poor Exception-Handling

‣ Overwhelming the resources

‣ Hard support for individual multithreaded tasks

• Unequal tasks

• ^Scheduling

• Cancelation

‣ No lifecycle for execution itself

48

(49)

Executors

ExecutorService

Thread Thread Thread Thread Thread ThreadFactory

Queue

Fixed Pool Size

ExecutorService

Thread ThreadFactory

Queue

Pool Size = 1

ExecutorService

ThreadThread

ThreadFactory Queue

ThreadThreadThreadThreadThreadThreadThreadThreadThreadThreadThreadThreadThreadThreadThread ThreadThreadThreadThreadThreadThreadThreadThreadThreadThreadThreadThreadThreadThreadThreadThread…

No Pool Size Additional Executors:

• ScheduledExecutor,

• CompletionServiceExecutor

…

(50)

Stopping Threads

‣ ^...

50

(51)

There is no safe way to stop a thread

public class PrimeGenerator implements Runnable { @GuardedBy("this")

private final List<BigInteger> primes = new ArrayList<BigInteger>();

private volatile boolean cancelled;

public void run() {

BigInteger p = BigInteger.ONE;

while (!cancelled ) {

p = p.nextProbablePrime();

synchronized (this) { primes.add(p);

} }

}

public void cancel() { Thread.stop(); }

public synchronized List<BigInteger> get() { return new ArrayList<BigInteger>(primes);

} }

(52)

Flag?

52

@ThreadSafe

public class PrimeGenerator implements Runnable { @GuardedBy("this")

private final List<BigInteger> primes = new ArrayList<BigInteger>();

private volatile boolean cancelled;

public void run() {

while (!cancelled ) {

p = p.nextProbablePrime();

synchronized (this) { primes.add(p);

} }

}

public void cancel() { cancelled = true; } public synchronized List<BigInteger> get() { return new ArrayList<BigInteger>(primes);

} }

(53)

Producer / Consumer ?

class BrokenPrimeProducer extends Thread {

private final BlockingQueue<BigInteger> queue;

private volatile boolean cancelled = false;

BrokenPrimeProducer(BlockingQueue<BigInteger> queue) { this.queue = queue;

}

public void run() { try {

while (!cancelled)

queue.put(p = p.nextProbablePrime());

} catch (InterruptedException consumed) { } }

public void cancel() { cancelled = true; } }

void consumePrimes() throws InterruptedException { BlockingQueue<BigInteger> primes = ...;

BrokenPrimeProducer producer = new BrokenPrimeProducer(primes);

producer.start();

try {

while (needMorePrimes()) consume(primes.take());

} finally {

producer.cancel();

} }

(54)

Interruption?

54

class PrimeProducer extends Thread {

private final BlockingQueue<BigInteger> queue;

PrimeProducer(BlockingQueue<BigInteger> queue) { this.queue = queue;

}

public void run() { try {

while (!Thread.currentThread().isInterrupted()) queue.put(p = p.nextProbablePrime());

} catch (InterruptedException consumed) { /* Allow thread to exit */

} }

public void cancel() { interrupt(); } }

(55)

Interruption

‣ Necessary Interruption Policy (what should happen if thread gets interrupted)

• freeing connection, resources

• ^unblocking

• keeping invariants

• cleaning up

‣ Tasks own not the thread (with the usage of thread pools),

policy mostly based on „get out of the way“

(56)

Interruption

What to do if InterruptedException is catched?

‣ Propagate the Interruption

‣ Restore the internal state

56

public Task getNextTask(BlockingQueue<Taskgt; queue) { boolean interrupted = false;

try {

while (true) { try {

return queue.take();

} catch (InterruptedException e) { interrupted = true;

// fall through and retry }

}

} finally {

if (interrupted)

Thread.currentThread().interrupt();

} }

(57)

Interruption of other threads

private static final ScheduledExecutorService cancelExec = ...;

public static void timedRun(Runnable r,

long timeout, TimeUnit unit) { final Thread taskThread = Thread.currentThread();

cancelExec.schedule(new Runnable() {

public void run() { taskThread.interrupt(); } }, timeout, unit);

r.run();

}

(58)

Future.cancel

58

public static void timedRun(Runnable r,

long timeout, TimeUnit unit) throws InterruptedException { Future<?> task = taskExec.submit(r);

try {

task.get(timeout, unit);

} catch (TimeoutException e) {

// task will be cancelled below } catch (ExecutionException e) {

// exception thrown in task; rethrow throw launderThrowable(e.getCause());

} finally {

// Harmless if task already completed

task.cancel(true); // interrupt if running }

}

(59)

Producer / Consumer, Poison Pill

Void call()

Producer

BlockingQueue

Void call()

Consumer

Item Item Item Item Item Item Item Item Pill

(60)

Executor Shutdown?

60 public class LogService {

private final ExecutorService exec = newSingleThreadExecutor();

...

public void start() { }

public void stop() throws InterruptedException { try {

exec.shutdown();

exec.awaitTermination(TIMEOUT, UNIT);

} finally {

writer.close();

} }

public void log(String msg) { try {

exec.execute(new WriteTask(msg));

} catch (RejectedExecutionException ignored) { } }

}

(61)

Design Patterns & Concurrency

Sebastian Graf, Oliver Haase