Is there a plan for adding IImageClient

[MikeAlhayek]

Hello,

The IChatClient interface is a great tool for integrating with AI providers. However, from what I can see, it doesn’t support interactions with image generation models like DALL·E.

Is there any plan to introduce an IImageClient or a similar interface that would enable communication with image generation models?

[stephentoub]

cc: @luisquintanilla

[luisquintanilla]

Hi @MikeAlhayek,

Multiple modalities is something we are looking to enable and are actively working on.

Currently the team is working on speech-to-text, which you can learn more about in this PR. #5838.

Image is one of those modalities we're tracking, but don't have a planned release date at this time.

[MikeAlhayek]

Thank you @luisquintanilla! Hopefully we see IImageClient being added sooner than later.

[akordowski]

I would like to follow up on @MikeAlhayek's question. I am trying to understand how Microsoft.Extensions.AI correlate to SemanticKernel. As I understand it the equivalent of the SK IChatCompletionService is the IChatClient. Therefore for the other SK services (like IAudioToTextService, IImageToTextService, ITextToAudioService and ITextToImageService) they may be in the future respective clients.

Currently only the SK IChatCompletionService can be utilized by Microsoft.Extensions.AI as IChatClient.
Am I right?

[luisquintanilla]

Embedding generation as well.

In Extensions.AI, you can use IEmbeddingGenerator in a similar way as IEmbeddingGenerationService from SK.

https://learn.microsoft.com/en-us/dotnet/ai/quickstarts/build-vector-search-app?tabs=azd&pivots=openai

[akordowski]

Thanks for your response. My questions was rather regarding the clients. Am I right that currently only the IChatClient is equivalent to IChatCompletionService and the other clients may come later for the other services?