Ask BeerGeek – or how Nitor Core time helped make Azure AI application development faster and easier

Before I delve deeper into quick AI prototyping and how we apply this technique at Nitor, I want to give a brief rundown of Nitor Core, one of our founding values. In short, Core time affords Nitoreans the opportunity to allot 10 percent of our work time to projects that will enable one’s personal growth as a digital engineer, and thereby make the company better with gradually growing expertise. Read more about Nitor Core here.

Laying the technical foundations

Azure AI and OpenAI services are generally located in different regions. Many of the OpenAI models are scattered around the world, and not all models are available in every region.

For efficiency’s sake, we’ve decided to centralize this by adding our own API layer in front of various AI services. This means the user doesn’t need to take note of where the models are deployed or be otherwise versed in Azure’s internals for effective functionality. All that is needed is the Nitor AI API we built. We handle all routing from our end.

A neat feature of this architecture is the fact that you can add to the production workload by adding more Azure OpenAI instances when needed. This helps us be creative with load balancing the services: If we are close to exceeding rate limits for GPT-4o, we can simply provision more instances in other regions to maintain availability under load.

Safe and smooth data travel in and out of AI applications

The Nitor AI API enables us to quickly create AI solutions with knowledge bases built with custom data. We can quickly upload data like a website’s contents with a simple data request, after which the API will scrape the content, create text from HTML files, partition the data, and create data chunks. This makes the process much easier to manage. All this is naturally highly customizable, depending on each client’s needs.

Azure AI Services offer a great starting point for many solutions, but sometimes the productization choices don’t align with cutting-edge needs we may have when prototyping GenAI capabilities. Our AI API layer enables us to pick and choose the best services from amongst Azure, other clouds, or custom-deployed open source services. We might start a custom knowledge base prototype with Azure AI Search and its Retrieval Augmented Generation (RAG) capabilities, but for more advanced work we can easily pivot to more customizable open source technology such as Microsoft’s Kernel Memory solution.

Responsible data usage practices with AI are a top priority for us, so the flexibility to choose where we keep the data is a key capability we gain from our AI API Layer. The data location is transparent to solution development which enables a quick pace for developers.

The API also provides an Ask endpoint for queries using natural language. The API will search for applicable content in the custom knowledge base and provide the user with appropriate answers with the help of the chosen large language model.

How BeerGeek became ProductBot – and why it matters

We created the first practical application during Nitor’s annual retreat last September, where we prototyped the API’s functionality by creating a mockup website called BeerGeek. As the name suggests, BeerGeek is a brewery encyclopedia that answers questions about beer. If I were to ask BeerGeek about the difference between IPA and lager, it would utilise our Ask endpoint and give us a compact, preformatted answer. I prefer lager, but to each their own.

The most rewarding example, however, was a piece of software called ProductBot created by two of our Agile coaches, who don’t work on code daily. A colleague following many thought leaders on LinkedIn was looking for a way to store all that accumulated insight and create questionnaires and other content from that. BeerGeek provided the launching pad for what became a concrete answer to this need.

The key takeaway was that ProductBot was created in a day, virtually requiring no coding experience. Enablers such as our AI API and underlying Azure AI Services are making it easier to bring good ideas into fruition as practical applications quickly — and in prototyping scenarios with only a basic understanding of API functionality.

Our custom API for Azure AI applications can handle the heavy lifting — all you need is an idea to start with.

Nitor focuses on evolving software solutions – get in touch and let us help your business reach the next pinnacle of software development