SOA + µservices (Tue Apr 12, lect 23) | previous | next | slides |

Look at applications of microservices and SOA in building scalable systems. Consider risks and benefits.

Logistics

Service Oriented Architecture

  • Build a system from a collection of services
  • Services talk to each other over the internal network
  • http, queueing, REST, or similar communications schemes
  • Services that are internal to the system
  • Not designed as an external API but as a way to structure a system
  • Contrast with Rails “Majestic Monolith” approaches
  • Big design and architecture question is how many services to have
  • What is the “right size”?
  • A very important pattern, but not uncontroversial

SOA vs Monolith

Benefits of SOA

  • Isolation
    • Each service is its own code base, database, and team
    • Each can be tested separately
    • Makes “scaling out” more possible
  • Robustness
    • If one service fails or errors out, it shouldn’t affect the others
    • Analogous to encapsulation in OOP
  • Scalability
    • Services become the unit of scalability
    • We can run 1,2,3 or more of the same service
    • A service can be moved to another server
  • Flexibility and Interoperability
    • No service can know or depend on how another service is implemented
    • Services will interoperate without worrying about the language or platform
    • Or it could be reimplemented in another language or another platform
  • Reuse
    • A service that is well designed can be reused in different contexts
    • There are services that different applications or products will require
    • Or a service designed by one group or company can be reused

Mechanics

  • Describe the “API” of the service
  • Usually an HTTP request and response
  • Long running services should be implemented as multiple calls
    1. One to initiate
    2. One to request results if any
    3. So as not to “block” the caller
    4. But then, what happens if the service has crashed or is hung?
  • Synchronous calls need to be very fast
  • Another excellent way to communicate is with a queue
  • Sinatra is especially suited for SOA services because it is lightweight

Discussion Questions

  • How would you recognize a system that is designed in an Service Oriented Architecture?
  • What are some of the advantages and disadvantages of this approach?
  • What is in an http response? And what would be somewhat different about the http response produced from an SOA system?

Monolith

  • What does a classic “Monolithic Web App” look like?
  • Web Server - Processes HTTP and delivers it to app code (Puma + Sinatra, for example)
  • App Server - (Your code, for example)
  • Database Server - Postgres, Oracle, Mongo, etc.

Advantages and Disadvantages

  • Advantages
    1. Good for small-medium projects with similar teams
    2. Single code base
    3. Don’t have to design, write, test, maintain service Apis
  • Disadvatages
    1. If teams are large there can be too much coupling
    2. Scaling out (if you need to) is very tough

Service Oriented Architecture

  • Single application, designed as
    1. Suite of (small?) services
    2. Each running in a separate process
    3. Communicating with lightweight mechanisms
    4. Relying on internal REST Apis or Queues or Both
  • Scaling up vs. Scaling out
  • Beware of premature optimization!

Conways law: Organize around business capabilities

Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization's communication structure" (see [Conway's Law](http://www.melconway.com/Home/Committees_Paper.html))

Communication

  • Trade-offs in communications between services
  • “Smart endpoints and dumb pipes” - End to end principle
  • What would “Smart pipes and dumb endpoints” mean? (see POTS)

“Unix Philosophy”

  1. Write programs that do one thing and do it well.
  2. Write programs to work together.
  3. Write programs to handle text streams, because that is a universal interface.

ps aux | grep ruby | grep -v grep | ruby -n -e 'puts $_.split[1]' | xargs kill

Famous (and valuable) Law: “Postel’s Principle”

Postel’s Principle

Postel's Principle: [designers] should follow a general principle of robustness: be conservative in what you do, be liberal in what you accept from others

The Big Picture

Evolutionary Design

  • Start with 1 big service (Majestic Monolith)
  • Identify with practice what sub-component could be a service
  • Keep things that change at the same time together in the same service (module)
  • Horizontal Scalability
  • Redundancy

Once services are introduced

  • Service discovery
  • Errors and timeouts etc
  • Load balancing (Smart client, Hardware, Software)
  • System level caching: Redis
  • Queueing

Databases

  • Monolith: one database with many tables
  • Microservice: Separate and distinct databases, per service
  • Shard or Partition

Interfaces

  • Internal implementation of service is irrelevant and isolated

Benefits and Goals that are satisfied:

  • Any service can be replaced by new code with (no/minimal) impact on other services
  • Each service can be designed and tested independently from the others
  • Failure in one service is kept isolated

Things to think about

  • Services need to be “discovered”
  • Services can be offline (crashed)
  • Permanently or temporarily
  • Handling of errors and time-outs
  • Monitoring and troubleshooting
  • Disaster Recovery
  • Timeouts on Services
  • What does a timeout mean?
  • Retrying with backoff
  • Circuit Breaker

Scenario

  • Service 1: Contains sinatra, processes http, displays forms etc.
  • Service 2: Executes certain long running services (e.g. major data load), fed by a job queue
  • Considering Service 3

When user tweets, how do we get that into the database?

  • Directly within Service 1
  • In Service 2, as a job on a general queue
  • In a new service 3, in a job queue, dedicated to tweets
  • In a new service 3, in a publish/subscribe queue

For Further Reading

Thank you. Questions?  (random Image from picsum.photos)