Scalability, continued (Thu Mar 3, lect 12) | previous | next | slides |

Examining what scalability means

Logistics

  • Secret code
  • Please do this quick survey
  • 10 Minute Quiz: xxxx

Scaling

Thought experiment

  • Real world example:
    1. Covid Test service in Shapiro
    2. Several failure modes
    3. Question: how to solve these?
activity discuss with your neighbors: if you were the manager, how would you address the problem?

Optimization:the search for bottlenecks.

  • What’s a bottleneck? Refer back to the example.
  • Moving target:
    1. When you eliminate/improve one bottleneck, it just reveals the next one.
    2. You make starting the dashboard activity faster….so that now you can notice that drawing the map overlay is slow.
  • Important: Measurement
  • Worse sin: Optimizing early. Why?

Performance

  • Performance is what what a user experiences as “slow” or “fast”
  • Response time to an operation initiated by the user
  • Perception!
    1. Can you ‘fool’ the user into thinking the app is faster than it is?
    2. Feedback: spinners etc
    3. Anticipation: start doing work before user requests it
  • Different (but intertwined with) scaling

Scaling

  • “How many X per minute can you do” (e.g. user log ins, page refreshes, notifications,…
  • How many (users, sessions, videos, pictures, etc) does the site need to support
  • Different from response time: “How long does it take to accomplish Y?” Related but different
  • Scaling has to do with the load on the servers
  • Big challenge: how fast or slow will the site or app grow?
  • Architectural techniques apply equally
    1. scaling up vs. scaling out
    2. caching
    3. load balancing
    4. database partitioning and sharding
    5. asynchronous processing

Patterns of scaling problems and solutions

  • “Clients” = web browsers accessing the site, mobile apps accessing the site, etc.
  • Load on the servers. Some scenarios, one or more of:
    1. Too many clients asking the server to do operation O 1. Individual clients asking the server to do operation P too often
    2. Operation Q is time consuming for the server to satisfy
  • Solutions can be
    1. Add an identical server to handle operations O, P or Q
    2. Send operation O to one server and operation P to another server
    3. Why are so many clients asking for O? Can we reduce the number?
    4. What’s the reason why a client would ask for operation P so often? Can we reduce that?
    5. Is there a way to make operation Q faster to satisfy?

More topics

Thank you. Questions?  (random Image from picsum.photos)