Microsoft Build 2023

Intro

Welcome to RoguePlanetoid podcast where each episode you can find insights for developers, whether you are a beginner or an experienced professional for Microsoft and related platforms, along with related technology and so much more to help Keep Current, Keep Coding.

Welcome

I'm Peter and welcome to episode five of the RoguePlanetoid Podcast about Microsoft Build 2023 which is the annual conference for developers with the main theme being on AI. Innovation is happening at a scale and pace like never before thanks to the infusion of AI across a broad spectrum of products and services, I'll be talking about the keynotes and sessions I watched both days but if you want to see them along with the rest of the sessions that were available to watch online then you can go to build.microsoft.com or check out the link in the show notes.

Day One

Microsoft Build Opening

Satya Nadella opened the first day of Microsoft Build 2023 with his keynote where he welcomed everyone there in person and being back together as it had been a few years since that had been the case, along with thanking those joining online like me and said that something big is shifting for developers. Sayta stated we have been pursuing the idea of the dream machine which first started from the “As we may think” paper that was published in 1945 through to the invention of personal computers and the world wide web up to today with ChatGPT. Satya quoted Steve Jobs who said “Computers are bicycles for the mind” Satya then added that recently it seems we have went from the bicycle to the steam engine.

Satya mentioned how we build is fundamentally changing, you can create a development environment quicker than ever and with things like GitHub Copilot to help assist developers and GitHub Actions to help stay on task and stay in the flow. Every week there is something new and Microsoft are infusing AI across every layer of the stack which started with GitHub Copilot to other products such as Microsoft 365, Bing and Edge.

Windows Copilot is bringing AI to the largest canvas of all and make every user of Windows a power user and will be fully integrated into Windows 11 where it could recommend a playlist on Spotify or simplify workflows within Windows and help users get more done and access all of Microsoft using Windows Copilot. Copilot Stack will allow apps with plugin extensibility which will allow developers to bring their applications to Copilots with common extensibility across services such as ChatGPT, Bing Chat, Microsoft 365 Copilot and Windows Copilot. Windows Copilot will change how you use your PC forever, invoked from the taskbar to show a side pane where you can ask questions with great suggestions you can put into action.

Microsoft are quickly delivering on their vision of bringing Copilot features to the web which includes bringing Bing to ChatGPT which enabled up-to-date content with citations along with bringing plugins that can be surfaced to places such as the web in Bing, assist with documents in Microsoft Word or for managing tasks in Microsoft Teams to deliver incredible productivity. There are already over fifty plugins available that can be used with Copilots and Microsoft are looking forward to what developers will build next and provide developers with more opportunities to reach more users.

Azure AI Studio is the full toolchain from Microsoft to build AI apps and Copilots, you can build your own models or ground them with your own data with support for retrieval augmented generation along with being able to create prompt workflows with safety built in.

Azure AI Safety is a responsible AI dashboard and includes model monitoring and content safety along with support for media provenance used in Bing and Designer for watermarking images and can also do audio watermarking.

Microsoft Fabric is a data analytics platform for the AI era with unified compute, storage, experience, governance and business models and this unification will fuel the next generation of AI.

Satya Nadella finished his opening keynote by saying that this year's Microsoft Build will show developers how they can build these next generation of applications and build them with safety first. The things that developers build can make a different to eight billion people not just a small group with diffusion that takes days and weeks not years and centuries and need to do this in such a way that we manage the energy transition required and empower every person on the planet to achieve more and what we do in the next weeks and months will have a profound impact on those eight billion people.

It was great to hear Satya Nadella open Microsoft Build with his vision, to provide the tools that they are developing to allow solutions to be created that may impact the entire world.

The era of the AI Copilot

Kevin Scott then followed by talking about the era of the AI copilot where he said that a lot has changed over the past four years but more has changed over the past year, from wanting to do things that he didn't know were possible to finding ways to make something that was impossible, possible thanks to the power of AI and do something great with it. Microsoft have the most powerful super computers, most capable foundation models from open-source and have the world's best AI infrastructure with Azure, which is the cloud for AI and the beginning of the end-to-end infrastructure for AI which is allowing Microsoft to deliver their AI powered solutions.

Greg Brockman from OpenAI then spoke about ChatGPT and going on to say that it's adoption had been huge but the challenge to build it had been immense, they had the idea of a chat system for years but the moment when it truly clicked was when they developed ChatGPT, they had the infrastructure from earlier models, but the training they did meant it could follow instructions and have a conversation but it wasn't designed for chat - but it wanted to chat. With GPT-4 they went back to the drawing board and rebuilt the infrastructure and wanted all the details to work correctly, it was sometimes the boring engineering work that often leads to success. In this field technology is clearly getting better but what developers can do is figure out how to make this work in their domain, understand what pain points there are to adopt this kind of technology and developers will be the people who will make AI great.

Kevin Scott then spoke about when developing their Copilots they realised the idea of them is pretty universal and applies to more things that just software development like with GitHub Copilot, so they needed to find out what was common and what stack would be needed and have been able to build a stack that allowed them to move quickly but with safety. Microsoft can provide developers the chance to create things that otherwise wouldn't exist. Foundational models can't do everything so need to have ways to be able to build on top of this technology to augment an AI application or Copilot to do more.

Kevin then spoke about the Copilot Stack which has the front-end for plugin extensibility and user experience, then there is orchestration with prompt filtering, meta prompt, grounding where you could have additional context provided such as retrieval augmented generation and plugin execution when going back up from the model and perform multiple passes to get what you need. Foundational models can be used or finetuned and there will be a marketplace of models you can use, or you could even train your own models.

It was great to hear from Kevin and Greg about how developers will work with AI to create their own solutions or augment existing ones and to see where this potential may lead.

Next generation AI for developers with the Microsoft Cloud

Scott Guthrie then spoke about next generation AI for developers with the Microsoft Cloud where he said that every app will be reinvented with AI and will see new apps build with AI that weren't possible before and Microsoft will make it easy to build these solutions using Azure, GitHub and Visual Studio. GitHub Copilot allows developers to create applications more easily and GitHub Copilot Chat will go beyond code suggestions and provide additional context such as asking it to make code more readable, easier to understand, perform refactoring and even ask it to fix the bugs in your code.

Scott then spoke about Microsoft embracing an open Plugin standard and have a developer experience to enable creation of these, including JSON files for meta data to where they can be installed with one plugin extensibility model for ChatGPT, Bing Chat, Microsoft 365 Copilot, Power Platform Copilot, Dynamics 365 Copilot, Windows Copilot and your own Azure AI Copilots. Microsoft are able to have so many Copilots as they build them on top of Azure AI which includes AI capabilities such as ChatGPT and GPT-4 and are the same ones that can be used by any developer, you can bring your own data and an Azure OpenAI instance is isolated from every other customer and is not used to train the foundational model with data being protected by the most robust compliance and security controls in the industry.

Azure AI studio allows you to ground AI models with your own data, including using retrieval augmented generation or RAG. Prompt Flow in Azure AI orchestrates AI models, prompts and APIs with support for prompt tuning and works with thousands of AI models from a model catalogue. Azure AI Content Safety can detect and assign severity scores to unsafe content and can work on human and generated content and allows for prompts to be tested and make sure these are filtered based on the needs of the application.

Azure is the world's AI super computer, it is the infrastructure powering ChatGPT and used to train the models, Microsoft have the largest AI training centres in the world and offer Hopper nVidia GPUs with Infiniband connections which are unique to Azure and they will be adding 120 more datacentres this year and Microsoft are committed to 100% renewable energy in the next couple of years for their data centres and AI infrastructure and by 2030 be carbon negative by removing all the carbon Microsoft has ever produced since their founding in 1975.

It was great to hear Scott talk more about the infrastructure that not only powers Microsoft's AI solutions but will also allow developers to create their own using the same tools and infrastructure, it was also amazing to see GitHub Copilot X be able to fix bugs in the code by just asking it to do exactly that. This ended the keynotes for the opening day and it was on to the sessions.

Getting started with generative AI using Azure OpenAI Service

The first session I was able to catch was getting started with generative AI using Azure OpenAI service.

ChatGPT has had unprecedented growth taking two months to reach 100 million users showing how useful it is in people's daily lives. Azure AI is building on these innovations so they can be integrated into applications and allow developers to build applications on AI. OpenAI has large pretrained foundational models such as GPT-3, DALL-E 2, ChatGPT and GPT-4 that are custom tuneable with your parameters and data and helps light up use cases that were not possible before. GPT-4 is achieving human-level performance in text generation with improved alignment and can generate complex documents, be tuned and steered with nuanced instructions and can instruct and annotate in any language, slang or dialect. Azure Open AI service is allowing for integration in interactive learning experiences to help people learn or help people be more effective in what they do and help them use their time more effectively. Innovations are providing more control on how you integrate Azure OpenAI in your solutions. Chat Completions API is a versatile interface for use in all scenarios not just chat and includes responsible AI steering. You may want to reason over long documents and token limits may not be sufficient so can have a vector database to retrieve relevant data and construct a prompt at runtime where you can combine a model with your own data and build an assistant that reasons over your own data.

Reinventing large language models with your own data is one of the key challenges in AI and the aim of Retrieval Augmented Generation or RAG is to separate a large language model or LLM from an externalised knowledge base using an orchestrator to mediate this. Plugins allow for the expanding potential as there's challenges to be addressed including accurate translation for a wider range of languages including support for Asian and African languages. Configurable content filters work outside of the core models by running both prompt and completion through an ensemble of classification models aimed at detecting and preventing output of harmful content. Azure OpenAI service supports provisioned throughput for high volume production workloads, offering predicable performance and reserved processing capacity available to meet demand along with cost savings for high throughput workloads vs token-based consumption.

It was interesting to hear about the features and services available to developers to make the most of AI along with ensuring safety from the very start!

What's new in .NET 8 for Web, frontends, backends, and futures

The next session was about what's new in .NET 8 for web, frontends, backends and futures. .NET is the complete solution for building modern web applications for front-end and back-end. You can have apps that serve content for requests such as MVC and razor Pages from ASP.NET with server-side rendering and can use Blazor for client-side rendering. New will be Blazor United where you can have server-side rendering where Blazor can update pages and intercept requests and update the DOM while still doing server-side rendering, but this may delay the output of a page so you can have streaming rendering with placeholder data and when a request has completed the page will be seamlessly updated, so something is still rendered so you get pixels on the screen as quickly as possible.

If want to add more interactivity you can apply render modes which can be server based and enable some interactive logic, you can get islands of web server interactivity in an application that can be freed up when no longer used. You can also change the component render mode to WebAssembly and then can have dotnet.wasm downloaded on a per page or per component level which is cached to be used later or you can set the render mode to Auto so could start users on web server when loaded for the first time and download the dotnet.wasm in the background and the application can use this when it has been downloaded.

.NET 8 also has a lot of focus on identity and authorisation as well as APIs, where minimal APIs can support form handling which can be useful for file uploads along with support for the problem details service which is a standard way of returning errors from a RESTful endpoint. Visual Studio now includes an API explorer where you can test your API endpoints along with helpful analysers that tell you when you are making a mistake in a minimal API. Servers and middleware features include request time outs so can define a policy that will issue a cancellation token you can deal with and short-circuit middleware can return a particular response directly. Authentication and authorisation improvements have been enhancements for Single Player Applications including client-friendly endpoints for identity managements, support for tokens in self-host scenarios and to remove Identity Server from templates and have identity manager which are APIs to manage an identity with an identity store and endpoints added so client UI can interact with these.

It was great to see what is in store for Blazor especially with ability to have hybrid client-server experiences and glad to see there's a lot of investment in .NET 8 and something developers are able to make good use of when creating web applications and more.

What's new in C# 12 and beyond

My final session of the first day was what's new with C# 12 and beyond. This covered the introduction of primary constructors to classes, although unlike records which also introduced them, these don't automatically add any properties but can use them as parameters throughout an instance of a class. Another enhancement was when creating lists or arrays with a new collection literal using square brackets around the items and the target type is inferred and can use an empty set of square brackets for an empty list or array and uses the same syntax introduced in pattern matching. C# 12 will introduce a new field keyword to an auto implemented property to allow you to write accessor implementations to read or write the underlying field but can keep the rest auto implemented, this is yet to be added as there may be something in scope called field but a solution may be to use this value if it is present otherwise perform the new behaviour, this functionality won't be implemented without introducing a breaking change and make sure to give a good experience to those affected and not impact everyone else in a negative way. Extension methods were introduced in C# 3 and allow for code to be layered differently to augment a class without changing the original class, but you can only do this with instance methods. There are plans to add an explicit extension which can be declared to add methods to perform functionality, like a transparent wrapper, this could be something you do something specific with and could have different methods that do something different. There are also planning implicit extensions where all objects will have extra members so can have an extension everything, where you don't need to wrap a new thing on the old thing, but it is the same object underneath. This new thinking around extensions hadn't been fully decided but could deliver the power to do separation of concerns and layering of software to give the next breakthrough in software composition.

That session was a great way to end the day with the sessions I could catch from here in the UK ready for the next day of keynotes and sessions from Microsoft Build 2023 which I'll also go into, you can read my article about Microsoft Build 2023 Day One at rogueplanetoid.com/articles/microsoft-build-2023-day-one or check out the link in the show notes.

Promo

Each week on Scottish Digest we'll tell you about a different place to visit in Scotland. We'll also tell you what you can do and see while you're there, we'll give you personal recommendations of places to eat and stay, and we'll even tell you about events and festivals that will be taking place. So, join us on each episode for a wee slice of Bonnie Scotland.

Day Two

Shaping the future of Work with AI

Day two of Microsoft Build 2023 started with the keynote of shaping the future of work with AI. Graphical User Interfaces made computers more accessible to more people and AI is making them even more accessible and more powerful to transform how people work in society. Microsoft has many productivity tools to help people get their work done, Microsoft 365 Copilot can help shape work by putting users in the centre and is more than just taking the most powerful foundation models but to ground this with data from the user's context with Microsoft Graph. Microsoft 365 Copilot will be available where users are such as Word or Teams and allow users to summarise documents or fetch information where needed and take actions on their behalf when needed. Extensibility with plugins allows developers to customise the user experience and allow a Copilot to take more actions on the user's behalf with these plugins using an open standard to support many different AI experiences. You can already create plugins with the existing Teams Toolkit for Visual Studio, first you need a RESTful endpoint that follows OpenAPI specifications then can select the methods you want to expose in the plugin, with the user interface of these plugins using Adaptive Cards which can be easily previewed using the tools.

Microsoft Graph is something that is added to every day by users, and this is the brains behind the Copilot's amazing reasoning capabilities, the data is not used to train the AI, it is used to ground it. You can add more value with Graph Connectors which can then provide more context and Copilot can use this and then provide citations to that content. Copilot could also interact with an enterprise application to retrieve or update assets with a message extension to work with these with context and invoke the correct actions that are required. Microsoft Syntex plugin allows you to structure and secure content for Copilot by leveraging your organisational knowledge, find, organise, and manage content along with being grounded in the user's context. It can automatically extract metadata and search or find relevant data with precision and generate new documents that can be reviewed or spotlighted with key details.

Panos Panay in the next part of the keynote talked about the future of Windows as there is one more place you can interact with over a billion users and that is Windows, it is an incredible time to be a developer, especially on Windows. For the devs already out there and the devs to be you might just be starting with AI and might not know where to start or may even be an expert already. No matter where you are on your journey, it is about Windows being there for you, it is not about going directly from the ground to the top, you need to take the steps along the way. Windows is fundamentally the absolute best endpoint for AI and the home of every one of your high ambitions for AI models. Not everyone knows how to get to AI so the first thing is to bring AI front and centre and bring those plugins forward to a billion people and bring this to the task bar to then make every user, a power user. Think about the history in a platform and then make it easier, you might not know a feature exists that can be surfaced but someone can just write it down and watch it happen or Windows can recognise you need to do something and then offer to do it.

The next part of the keynote covered some of the experiences that have been added to Windows including the need to support Bash on Windows which this led to development of Windows Subsystem for Linux or WSL and enabled a better experience with Windows Terminal which is now a top five supported project by developers. Then there is Dev Home which is a brand-new experience for developers to let you focus on what you are working on and easily install the tools and packages you need. Dev Home helps you by removing distractions and can get a dashboard which could contain pull requests or issues from GitHub along with being able to track tasks with support for extensibility. Windows Terminal itself will support new features including tab tearouts and Github Copilot X integration. You can also help engage customers with your app using Widgets where you can amplify your apps and services and there will be support for additional layouts for Widgets where can have a mix of News feed and Widgets or more space for Widgets. Microsoft Store will have AI generated review summaries to help fans influence their favourite applications or allow developers to get an overview of customer thoughts. AI can work for every developer, and you can get started with things like ONNX runtime and if you don't know where to get started there are AI models that be integrated into applications easily and to get to the next level you need. ONNX Runtime and Olive Toolchain can help create experiences that are tailored for your customers. Olive Toolchain brings all the optimisation steps you need and there are walkthroughs to help you get started and simplify a workflow such as optimising Stable Diffusion to reduce the time to generate an image. You can also create a hybrid loop between cloud and client where can use cloud to host large data sets but then can have an Olive Toolchain optimised smaller version of a model running locally. You can also accelerate the development of AI using GPUs and there are also NPUs which are powerful ways to accelerate AI workloads along with dedicated support for ONNX runtimes on hardware and there are more NPU devices coming to the market including those used on the Windows Dev Kit 2023. With Windows, AI enabled silicon, ONNX runtime and Olive Toolchain help make every developer an AI developer.

Finally in the keynote there were some general thoughts about the unprecedented change AI is bringing and that we need to think about the impact on our applications. AI is the new interaction technology, but will AI be beside, inside or outside your application. Beside is when you add Copilots to your application and keep the original application architecture the same but deliver immensely capable tools and experiences that were not possible before. Inside is to create apps with AI at the centre of the experience and completely redefine the experience such as taking pro-level experiences and turning them into one click with fewer toolbars and menus. Outside goes beyond context and is executed globally across apps and tasks and acts as an agent to go across workflows and devices. Solutions that are provided today allow you to develop for one or all of these, it may feel like you are learning again but Microsoft are alongside you, each of the solutions build upon the next and help reinvent your interaction model and make work more powerful and intuitive to help reach and empower more people.

It was great to hear the keynote and all the speakers from the opening of day two, especially Panos Panay who took time to step off stage and walk through the audience all while holding everyone's attention in a very compelling and engaging talk that was truly mesmerising, with the keynotes complete it would be on to the sessions I was able to catch from day two starting with developer joy with Scott Hanselman and friends.

Developer joy with Scott Hanselman and friends

Dev Home is the new home for developers in Windows and is fully open-source and extensible where any developer on Windows can find something helpful and also comes with a fully customisable dashboard. Also available is WinGet Configuration which lets you get into a desired state quickly as onboarding is a hassle and you want to get started as soon as possible. Dev Home wants to remove the developer toil and centralise the experience for developers which starts from the updated Out of the Box Experience on Windows 11 with a Developer option that installs Dev Home, once launched you can use a YAML file to install the things you need for your development environment such as Visual Studio, you can also customise the Dashboard to see system widgets to display information about CPU, GPU and Memory. You can also create a Dev Drive which can dynamically resize and supports the Resilient File System which is a workstation quality file system and offers improved performance when building projects.

It was great to see Dev Home and pleased to see that it is a Windows App SDK application which was great to see and there was more information to follow in the next session which was New developer experiences in Windows.

New developer experiences in Windows

Dev Home is the centralised location for all your developer needs with integrated machine setup flow and is available now in the Microsoft Store and GitHub and comes with the GitHub Extension installed by default and you can build your own extensions. When you first launch Dev Home you can connect to GitHub and go to the Dashboard and along with system widgets there are ones for GitHub including reviews requested, issues, pull requests and more allowing you to monitor what is important to you and can install other extensions, there will be an Azure Dev Ops plugin in the future.

Dev Drive is a new trusted drive with improved performance using Resilient File System or ReFS and has a performance mode for Microsoft Defender Antivirus that reduces impact on your workflow bringing the performance, security and control that can only be delivered by Windows. You can create Dev Drives with the size you need from fifty gigabytes and select the drive letter you need and when signed into GitHub you can clone repositories into Dev Drive automatically.

WinGet is the package manager command line tool for Windows and WinGet Configuration is a repeatable, reliable mechanism to get your machine to a code ready state as quickly as possible, including setup of languages, tools and settings needed for a project. Windows Terminal is being updated to be able to use chat with the new GitHub Copilot X. Updates to PowerToys include Peek, Paste as plain text, registry file previewer, hosts file editor and mouse without borders which allows you to use one mouse and keyboard on two separate PCs. Windows itself will be updated to support additional file compression formats such as 7zip tar and rar and Windows Settings will support the ability to hide the clock in the system tray and there will be a task bar setting to never combine task bar buttons along with an end task option for the context menu for applications in the task bar. Dev Home will be updated in future to support multiple dashboards and have deeper integration with repositories on GitHub and the Xbox team are bringing the public GDK for Xbox development to Dev Home.

Overview

Microsoft Build 2023 along with the public sessions also had other sessions you could participate in and one was Help us elevate your development experience on Windows which I participated in, unfortunately I cannot share any details from the session except to say it was an invaluable opportunity to give feedback about some features and participate more actively in a session, something that was really great to be a part of and marked the end of the sessions I caught up with during the conference, you can read my article about Microsoft Build 2023 Day Two at rogueplanetoid.com/articles/microsoft-build-2023-day-two or check out the link in the show notes.

Outro

Thanks for listening to the RoguePlanetoid podcast where you can find insights for developers, whether you are a beginner or an experienced professional using Microsoft and related platforms, along with related technology and so much more. You can also find any links in the show notes for this episode or visit rogueplanetoid.com/podcast for details of this episode and podcast. Keep Current, Keep Coding!