Stock Alerts Infrastructure

We’ve talked about the features that we’ll be implementing in Stock Alerts. Today we’ll look at the infrastructure that will be needed to support those features.

I’ve been working in Azure for several years now, both in my work life and on side projects. Being a primarily .NET developer, it makes sense that Azure is my preferred cloud. One of these days I will probably check out AWS, but for this project we’ll be hosting our backend services in Azure.

Microsoft Azure

More Than Just CRUD

When deciding what to build for this project, I wanted to do something that was a bit more than just a simple CRUD app that consists of an app talking to a web service talking to a database. Stock Alerts will need to continuously monitor the current prices of all stocks for which there are active alert definitions and evaluate whether the alert should be triggered, so we’ll need a process that runs on a regular basis to perform that work. Further, when the process detects that an alert should be triggered, it needs to send out notifications on the user’s preferred channel(s).

For this processing, we’ll use a combination of Azure Functions and Service Bus queues.

Here’s a sequence diagram depicting the retrieving of quotes, evaluation of alert definitions, and sending of notifications:

Stock Alerts Notification Sequence Diagram

Alert Definition Evaluation

The evaluation of the active alert definitions will have a predictable load. The system will query the stock data provider on a defined time interval for the distinct set of stocks for which there are active alerts and iterate through the alert definitions and evaluate them against the latest data received from the data provider for that stock.

A timer-triggered Azure Function, which is essentially a CRON job running in Azure, will work nicely for periodically pulling new stock data. Initially, there will be a single function instance to pull the data, but this work can be partitioned out to multiple function instances if/when the need arises. It will then enqueue a message on a service bus queue (alertprocessingqueue) for each active alert indicating that there’s new data and the alert needs to be evaluated.

A service bus queue-triggered function (EvaluateAlert) will receive the service bus message and perform the evaluation for a single alert definition.

Sending Notifications

The actual notification of users, on the other hand, will likely be characterized by periods of low activity with occasional spikes of high activity. Imagine a very popular stock like AAPL receiving an earnings surprise and opening 5% higher – several alert definitions could be triggered at once and notifications will need to be sent immediately.

Azure Functions will help us with this use case as well – we’ll enqueue notification messages on service bus queues (pushnotificationqueue, for example) when alerts are triggered and service bus queue-triggered functions (SendPushNotification, for example) will respond and send out the notifications. We’ll have a queue for each delivery channel (push, e-mail, SMS), and a function for each as well.

When AAPL spikes and 500 alerts are triggered, 500 messages will be enqueued on service bus queues (assuming each user only has one delivery channel) and 500 functions will be invoked to deliver those notifications.

The Infrastructure

So what Azure resources will be required to support the Stock Alerts features? Here’s a diagram of what we’ll need for the MVP:

Stock Alerts MVP Infrastructure Resources

We’ve got an Azure SQL database to store our users and their preferences, alert definitions and criteria, and stocks and their latest data.

We’ve already talked about the service bus queues, which are used for communicating between the Azure Functions, and we’ve already talked about the Azure Functions as well.

The Stock Alerts API will be an ASP.NET Core Web API service running in Azure, and it will expose endpoints to handle the user and alert definition maintenance as well as authentication.

The Stock Alerts web app, though depicted on the diagram, will actually be implemented post-MVP.

Current State

The above shows the infrastructure as I plan to have it at launch. Below is the current infrastructure I have deployed in Azure:

Stock Alerts Current Infrastructure Resources

All of the API endpoints are currently implemented as HTTP-triggered Azure functions. I did this because I already had the StockAlerts.Functions project, and I didn’t think there’d be that many HTTP endpoints. As I started implementing the authentication endpoints and I ran into some of the limitations of Azure Functions HTTP endpoints (i.e., you can’t inject logic into the pipeline as you can into the ASP.NET Core middleware for a full-fledged Web API), I increasingly felt like the API endpoints deserved their own project and app service. It’s on my TODO list to move these into their own project and service.

Wrapping Up

I think the most interesting part of the Stock Alerts infrastructure is the use of Azure Functions and Service Bus queues to evaluate alert definitions and send notifications. Azure Functions make sense for these processes because they can be triggered by schedule or service bus queue message (among other methods), and they are easily scaled. Service bus queues are appropriate for the communication between functions because they are highly available, reliable, and fast.

Though one of the key value props of serverless computing is automatic scaling, I don’t have practical experience with scaling Azure Functions during periods of high load. I’ll log and monitor timings from the various functions to ensure that notifications are being delivered in a timely fashion from when they are triggered, which is crucial for Stock Alerts’ primary function.

That’s all for now. Thanks for reading.

-Jon


If you’d like to receive new posts via e-mail, consider joining the newsletter.

Working on a New Side Project in Public

For the past month or so I’ve been working on a new side project. It’s a small project that will allow me to exercise existing skills, re-sharpen older skills that I’ve not used in awhile, and learn some new ones.

I’d like to share it with you…

Working in Public

I’ve been working my side project every weekday morning for an hour or two before my workday starts and on the occasional weeknight or weekend. I toil away mostly in solitude, except when I solicit feedback or ask for advice from a colleague and buddy who’s working on his own project.

I’ve recently been inspired by a few folks on various podcasts to start working in public. They speak of the benefits of sharing what you’re learning, developing, building with your audience as you’re working on it, an idea that strikes fear in the hearts of those who, like me, are inclined towards self-conscious perfectionism. It’s difficult to put your in-progress work out there for all to see, warts and all.

There’s been an increasing trend towards working in public, particularly among indie makers and developers. I dipped my toe into those waters with my last project, Open Shippers, earlier this year*, and I plan to get back in the pool with this project.

My plan is to post fairly regular (weekly?) updates about what I’m doing on the project:

  • What did I do the prior week?
  • Is there a particular problem I solved, pattern I used, or trick I learned that is worth sharing?
  • What do I plan to work on in the coming week?
  • Any particular challenges worth mentioning?

Things like that.

These will be pretty rough, unpolished posts. Sometimes a particular topic I encounter will be worthy of becoming a more in-depth, polished post, and put these up from time to time.

Goals

So what are my goals with working on this new project, and doing so in public?

  • Get into the habit of writing regularly.
  • Sharpen old skills and learn new ones.
  • Share what I’m working on and what I’ve learned. Hopefully it is helpful to someone.
  • Demonstrate the process of taking a product from concept to market.
  • Demonstrate well-architected, working product on the Azure / .NET Core stack.
  • Teach.
  • Learn.
  • Have fun.
  • Make some lunch money if anyone subscribes to the service.

What’s the Project?

Concept

The idea is not particularly novel or interesting, but it’s a good candidate for what I’m trying to achieve with this project.

I’m building a stock alerts app (hereafter referred to as Stock Alerts until I think of a good name for it) that will allow the user to create an alert definition for a given stock based on a set of criteria that they define. When the alert is triggered, the user will be notified according to their preferences by push notification, SMS, and/or e-mail.

“Aren’t there a ton of other stock alert apps out there?” Yes, there are. Some just let you set a simple price alert; others offer many more features. I plan to support complex alert criteria that include not only simple price alerts, but eventually also technical and fundamental alerts as well, which will differentiate the app from a portion of the market.

I also want to challenge the idea that you have to have a completely novel idea to succeed in building an app that people will pay for. There’s room for multiple competitors in most markets, and oftentimes your app will be the right choice for a segment of the market that’s not currently being served well.

To be clear, I have no illusions of replacing my income with this app, but it will be great fodder for achieving the goals I mentioned above, and it should be a fun little project.

Tech

Here’s the some of the technology I’m using:

Current Status

The backend API is coded and running in Azure. I’ll share more detail about the API and how it’s built in future posts, including the overall architecture. I’ve been working on the mobile app for about a week now. I’m able to register/login through the mobile app and display my current alert definitions. Last week I put together some UI wireframes for the Create Alert Definition screens, and I’ll be working on implementing those screens this week.

Wrapping Up

In the spirit of working in public, I’ve made the Stock Alerts repository public: https://github.com/jonblankenship/stock-alerts. As with most projects, there are areas of the codebase that need some work and refinement, but we’ll address those things together down the road.

Thanks for reading! Until next time,

-Jon

(* After spending several months working on Open Shippers, I was burned out on it by the time I launched it in preview in February. I failed to differentiate it from other similar services out there, and I didn’t have any gas left in the tank to work on building an audience and making it successful. Someday I may do a post-mortem on the project. Open Shippers wasn’t a total failure though – I learned quite a bit while working on it, particularly about ASP.NET Core and Blazor, and I’ve reused several things I developed for Open Shippers in more recent projects. The site is still live, and I may revive efforts around it at some point, but I’m content to work on something else for the time being.)


If you’d like to receive new posts via e-mail, consider joining the newsletter.