### Logistics * Secret code: * What is the highest number of threads you've gotten to? * How much of the test interface have you implenmented? * Any questions?
Scalability Pattern: Concurrency
Do more with the available resources
Do more things at the same time
Resource idle time is your enemy
CPU
Network
Disk
Database
Scenarios when concurrency is an option
Users can ask for a detailed report about their twitter traffic
It can be a one shot, or regenerated at midnight
It can be displayed on the screen or emailed
Inline is no good because of computational cost and error handling
User enters a new value in a spreadsheet
The cell can update immediately
The recalculation can be done concurrently (asynchronously)
Note: AJAX uses a very common form of concurrency
Note: Count the users’ own computer as one of the processors!
Scaling pattern: concurrency on a single computer
Difference between syncrhonous and asynchronous
Difference between concurrent and parallel
Processes (“forking”)
Use more memory (new VM for each process)
for the data + the program + everything
“Copy on write”
If parent dies before children, they can become “zombie” processes
Context switching very expensive
Communication expensive (IPC or file system)
Slower to create and destroy
Less hard to program and debug (not easy!)
Threads
Use less memory (Shared memory space)
All threads die when oricess dies
Context switching cheap
Communication cheap (via queues and shared memory)
Fast to create and destroy
Harder to program and debug
Ruby-related issues
Ruby and Pyton “GIL”
Global interpretter lock
Essentially they become single threaded
Except for asynchrony provded by OS via IO operations
“It’s complicated”
Ruby libraries that use processes
Resque - background processing framework
Unicordn - http server
Ruby Libraries that use Threads
Sidekiq - background processing framework
Puma - http server
Thin - http server
Thread-safe
A property of software, or a routine or a class
Does it behave ‘well’ when running in a thread (sharing memory)
Deadlock (“deadly embrace”)
Example with two people and two tools
Race Condition
When the results vary due to
How to avoid: using semaphors, queues, and other techniques
:slide_title
Let’s take a look at
Multithreaded Ruby
Let’s take a look at
Introduction to Concurrency Models in RUby
Scaling Pattern: Asycnhronous processing
Real examples
Account registraton confirmation emails (actually all emails)
Daily (periodic or episodic) notification emails
Automatic backups or archiving
Image Resizing
Spam checking
Synchronous: Caller
waits
for response
Asynchronous: Request returns immediately, but result comes later
Web server mainly responds to http requests!
Background processing happens even if no requests!
How do processes work in a modern Operating System?
Scheduler part of the OS
Processes can be fairly heavy weight
Let’s estimate how many background processs you would need
How would you handle processes that:
Were taking too long?
Had crashed?
Needed to be restarted?
Or what if the server crashes as a whole?
It’s a mess. Need abstractions!
Case 1: Schedule based
“CRON jobs”: Most elementary. Require “privileges”
Alternative: Native “scheduler” application (e.g. Heroku Simple Job Scheduler)
Case 2: Request based
Request handled ‘asynchronously’ of course
By local server, in a separate process
By a remote server
Request queueing
What happens when requests come faster than we can handle them
How do we add more performance in handling request
SOA API
Make sure the API returns right away and requestor has a way to check status
team
Plan to use asynchronous processing?
Thank you. Questions?
(random Image from picsum.photos)