Developing Windows Widgets with Generative AI
Windows Widgets
Overview
Windows Widgets enable access to at-a-glance interactive and dynamic information all from the click or swipe of the Widgets Board. Adaptive Cards are platform-agnostic JSON-based UI snippets for rich interactive content such as Windows Widgets.
Widgets
Windows Widgets enable quick-and-easy access to at-a-glance information without needing to open an app or website including built-in and third-party Microsoft Store widgets and can organise and save time from the Widgets Board with built-in Widgets such as Weather or third-party widgets from the Microsoft Store such as Messenger. Widgets can be customised based on your own interests and preferences or Widgets can be personalised to show what information is important to you. They can be interactive to allow engagement with content directly or they can be dynamic including real-time updates to text or images and are designed using Adaptive Cards which can updated partially with data binding or completely by another Adaptive Card.
Adaptive Cards
Adaptive Cards are platform-agnostic UI snippets in JSON for rich interactive content across platforms or services such as Microsoft Teams, Viva Connections and templates for Windows Widgets. Adaptive Cards are used to exchange at-a-glance content using JSON-based templates and support elements such as text or images along with layouts such as columns or tables. They can be created using the JSON schema or using the drag-and-drop designer to add or update elements and structure or can even be created using Generative AI. They can be can be updated dynamically based on user actions for a responsive experience or respond to changes to data with databinding to show up-to-date information. For more about Adaptive Cards you can read Designing Windows Widgets using Generative AI or for documentation, schema, designer and more visit adaptivecards.io.
Widget Provider
Windows App SDK
Windows App SDK is used to develop modern native Windows desktop apps that can use WinUI 3 targeting Windows 10 RS5 to the latest version of Windows 11. It is decoupled from Windows itself and is updated every six months with new features and functionality delivered via NuGet to create applications that can be developed using C++ with Win32 or C# with .NET which is Microsoft's modern open-source software development framework. Windows App SDK is installed with Windows application development workload in Visual Studio to create WinUI 3 in Desktop apps. You can find out more about Windows App SDK using C++ with Win32 or C# with .NET at aka.ms/winappsdk.
Widget Provider
Windows App SDK applications can become a Widget Provider by implementing key methods from IWidgetProvider and IWidgetProvider2 interfaces in the Microsoft.Windows.Widgets.Providers namespace. IWidgetProvider includes ActivateWidget and DeactivateWidget when the Widget Board is interested or not in updates from a Widget Provider, CreateWidget and DeleteWidget for when a Widget has been added or removed from the Widgets Board. OnActionInvoked is for when a Widget Action has occurred and OnWidgetContextChanged is when the size of a Widget has changed along with IWidgetProvider2 which includes OnCustomizationRequested for when a Widget accepts customisation in Widgets Board. You can find out more about developing a Widget Provider for Windows Widgets at aka.ms/widgetdevdocs.
Comentsys.Toolkit.WindowsAppSdk
Windows App SDK applications using C# with .NET can become a Widget Provider even more easily by leveraging Widget functionality provided by my Comentsys.Toolkit.WindowsAppSdk NuGet package. Developers can inherit WidgetProviderBase supporting the key methods from IWidgetProvider and IWidgetProvider2 along with AddWidget and UpdateWidget for managing or registering Widgets. Developers can not only create a Widget Provider with WidgetProviderBase but can also inherit WidgetBase to implement a Widget itself including returning the Adaptive Card template and JSON data. Comentsys.Toolkit.WindowsAppSdk wraps classes developed by Microsoft for Windows Widgets along with other useful functionality and developers can find and install it from NuGet or you can also find Comentsys.Toolkit.WindowsAppSdk including source code and example Widget via comentsys.com/packages/toolkits.
Generative AI
Overview
Generative AI reduces time and effort by employing AI models to generate new ideas, automate content creation or format information efficiently. It leverages AI models trained on extensive datasets encompassing text, images or various other content types and models learn patterns and structures within data to grasp the underlying context of the data. Generative AI can create new content based on its training by utilising learned patterns and context and output can even be fine-tuned by adjusting prompts to produce precise relevant results.
Azure OpenAI
Azure OpenAI provides developers with access to powerful language models from OpenAI such as GPT-4o to be integrated into their applications for content generation and more along with enabling the creation of innovative and state-of-the-art intelligent applications including natural language processing and understanding capabilities. Azure OpenAI Service offers REST API access and integration with Azure AI Foundry for further model exploration and fine-tuning of models using your own data along with allowing developers to leverage the full power of OpenAI models or experiment with these models using GitHub Models. Azure OpenAI is part of Azure at azure.net and GitHub Models which can be used for free experimentation at github.com/marketplace/models.
Microsoft.Extensions.AI
Microsoft.Extensions.AI simplifies AI integration for .NET applications by providing a unified set of abstractions for interacting with various AI services such as language models and embeddings. It eases the integration of AI services for developers across various providers by standardising API interactions to provide a single API throughout an application. Microsoft's objective is to provide standard implementations for developers including caching, telemetry, tool calling and other common tasks that work with any AI service provider. Implementations include OpenAI and Azure AI Inference for models such as GPT-4o that can be used from Azure OpenAI Service or GitHub Models and Llama with Ollama. Microsoft.Extensions.AI packages are available on NuGet in preview and library authors are encouraged to implement abstractions for other AI service providers.
Summary
Widgets provide easy-to-use dynamic or interactive access to information at-a-glance with build-in or third-party Widgets using Adaptive Cards which can be delivered by a Widget Provider for Widgets created using the JSON schema, drag-and-drop designer or using Generative AI. Widget Providers can be easily implemented in Windows App SDK and can even be powered with Generative AI which can be easily integrated into .NET applications with Microsoft.Extensions.AI implementations.
Conclusion
Developing Windows Widgets with Generative AI which you can watch now on YouTube for Microsoft Reactor or check out github.com/rogueplanetoid/developingwindowswidgets for Slides along with Code for a Prompt Widget that delivered dynamic content with Generative AI in a Windows Widget.