Anthony Chu Contact Me

Large-Scale Serverless Machine Learning Inference with Azure Functions

Thursday, September 5, 2019

This article is part of #ServerlessSeptember. You'll find other helpful articles, detailed tutorials, and videos in this all-things-Serverless content collection. New articles are published every day — that's right, every day — from community members and cloud advocates in the month of September.

Find out more about how Microsoft Azure enables your Serverless functions at https://docs.microsoft.com/azure/azure-functions/.

Azure Functions recently announced the general availability of their Python language support. We can use Python 3.6 and Python's large ecosystem of packages, such as TensorFlow, to build serverless functions. Today, we'll look at how we can use TensorFlow with Python Azure Functions to perform large-scale machine learning inference.

Overview

A common machine learning task is the classification of images. Image classification is a compute-intensive task that can be slow to execute. And if we need to perform classification on a stream of images, such as those coming from an IoT camera, we would need to provision a lot of infrastructure to get enough computational power to keep up with the volume and velocity of images that are being generated.

Because serverless platforms like Azure Functions automatically scale out with demand, we can use them to perform machine learning inferencing and be confident that they can keep up with high-volume workloads.

Without scaling, this is how slowly our app currently runs:

GIF

Continue Reading...


Async Streams with IAsyncEnumerable in .NET Core 3

Wednesday, July 31, 2019

One of the most exciting features of .NET Core 3 and C# 8.0 has been the addition of IAsyncEnumerable<T> (aka async streams). But what's so special about it? What can we do now that wasn't possible before?

In this article, we'll look at what challenges IAsyncEnumerable<T> is intended to solve, how to implement it in our own applications, and why IAsyncEnumerable<T> will replace Task<IEnumerable<T>> in many situations.

Check out all the new features in .NET Core 3

Continue Reading...


Experiment with New Languages and Frameworks with VS Code Remote Development in Containers

Monday, July 8, 2019

In April, Visual Studio Code launched their new Remote Development extensions. With VS Code Remote Development, a VS Code instance on our local machine can access and use a container, remote machine, or Windows Subsystem for Linux (WSL) as a full-featured development environment.

I've been using all three flavors of VS Code Remote Development and they each unlock some really exciting scenarios. Today, we'll look at how VS Code Remote with containers allows us to experiment with new languages and frameworks without installing any new tools or SDKs on our machines.

The specific scenario we'll look at is to use a container to test out the latest nightly build of ASP.NET Core 3.0 (currently Preview 7).

Continue Reading...


Event-Driven Serverless Containers with PowerShell Azure Functions and Azure Container Instances

Monday, April 29, 2019

Today, the Azure Functions team announced the public preview of their V2 runtime's support for PowerShell (PowerShell was previously in V1 as an experimental language).

In Azure Functions V2, PowerShell isn't just another language for writing serverless apps; it opens the door for many event-driven automation scenarios that weren't easily achieved with Azure Functions before. It uses a brand new PowerShell Core language worker built by the PowerShell team.

Using managed identities, a PowerShell Azure Function app can automatically log into a system-managed Azure Active Directory identity. When combined with the Azure PowerShell module, we can write Azure Functions that can securely manage any resource in Azure.

In this article, we'll look at how we can trigger and monitor batch jobs using PowerShell Azure Functions and Azure Container Instances.

Continue Reading...


Real-time Speech-to-Text and Translation with Cognitive Services, Azure Functions, and SignalR Service

Tuesday, March 26, 2019


When we do a live presentation — whether online or in person — there are often folks in the audience who are not comfortable with the language we're speaking or they have difficulty hearing us. Microsoft created Presentation Translator to solve this problem in PowerPoint by sending real-time translated captions to audience members' devices.

In this article, we'll look at how (with not too many lines of code) we can build a similar app that runs in the browser. It will transcribe and translate speech using the browser's microphone and broadcast the results to other browsers in real-time. And because we are using serverless and fully managed services, it can scale to support thousands of audience members. Best of all, these services all have generous free tiers so we can get started without paying for anything!

Overview

The app consists of two projects:

  • A Vue.js app that is our main interface. It uses the Microsoft Azure Cognitive Services Speech SDK to listen to the device's microphone and perform real-time speech-to-text and translations.
  • An Azure Function app providing serverless HTTP APIs that the user interface will call to broadcast translated captions to connected devices using Azure SignalR Service.

Architecture

Continue Reading...