Queueing (Thu Apr 7, lect 22) | previous | next | slides |

Queuing and message passing are basic techniques in scaling

Scalability Pattern: Queueing

SOA: Divide software into independent “services” that talk to each other
- Client: code that needs the service
- Server: code that implements the service
Previously: SOA required REST between servers

Discussion: Why is REST/HTTP between servers usually a synchronous request

Queueing is a fundamentally different way of creating services and concurrency
Syncrhonous:
1. Client makes a request (HTTP get) and waits
2. Server Receives request
3. Server does the service, during which client and server are waiting
4. Server sends response
5. Client is free to continue processing
Asynchronous
1. Client makes a request (‘Publish’ to a queue)
2. Client can now continue
3. Server picks next item off queue and does the service.
4. When service complete, notifies client to use the result.
That’s the theoretical framework

Discussion: There are at least 4 patterns that you can use to implement this. What are the tradoffs?

In nanoTwitter Logic:

Decide on the contents of the email, as a template (e.g. erb)
When a follow request is processed (during request handling)
- Populate the template with info about who is following who
- Retrieve the email address
- Initiate the sending of the email

How to initiate sending of email?

Inline:
- Write a method which does the work, which will cause the email to be packaged up and sent to the mail server. Once the mail server accepts the email, execution continues.
- Problems: Request handling is delayed and so the whole site hangs for each follow request: UNACCEPTABLE
Thread * Write a method that does the work (like above) make the mail server request in a thread.
Seperate SOA / REST Service
- Write a REST service (nanoTwitterEmailer) which accepts an EmailRequest and returns immediately to allow the main line code to continue. Email is sent to outside Mail server in a seperate process by nanoTwitterEmailer.
- Problems: How are error handled? What if the REST Service crashes, or hangs? What if it is flooded with requests? How do we keep track of the emails that need to be sent? What if the whole nanoTwitterEmailer service dies?
Message Queue
- Use an EmailRequest message queue to collect info about the email that has to be sent in it’s queue. Configure nanoTwitterEmailer to subscribe to a message queue and process the EmailRequests in the order they were submitted.
- Result: The EmailRequest message queue robustly tracks the email requests. If the nanoTwitterEmailer dies, then when it comes back up the requests are still there. If there are too many requests, it’s simple to run a second nanoTwitterEmailer.