Friday, November 4, 2022

DALL-E OpenAI Image Generator for Sitecore

On January 5, 2021, OpenAI revealed DALL-E, an artificial intelligence program that creates images from textual descriptions, on January 5, 2021. It uses a 12-billion parameter training version of the GPT-3 transformer model to interpret the natural language inputs and generate corresponding images. 

Since the initial private beta release, access to DALL-E was limited to those on a waitlist. However, about a month back, on September 28, 2022, DALL-E 2 opened to the public, and the waitlist requirement was lifted. Over the past month, millions of users have explored DALL-E's capabilities using https://labs.openai.com/.

Yesterday, on November 3, 2022, OpenAI announced the DALL-E API, which developers could start immediately consuming. 🤩


DALL·E joins GPT-3, Embeddings, and Codex in our API platform, adding a new building block that developers can use to create novel experiences and applications. All API customers can use the DALL·E API today.
As soon as I heard the news, I couldn't help but immediately explore the API myself. I was already familiar with OpenAI's API playground for text-based AI  prompts and had an existing API key ready. 

My first instinct to consume the DALL-E API was to use plain old Windows PowerShell 🤓

"A photorealistic close-up of a variety of river rocks"

"A super cute puppy"

So dope 😎 - and wildly simple!

DALL-E OpenAI Image Generator for Sitecore

AI-generated images were featured prominently in Paige O'Neill's keynote at the 2022 Sitecore Symposium in Chicago. 

There's real value in this technology. A reality where designers, marketers, and developers who need a patterned backdrop can unleash their creativity to quickly and easily generate an image without needing to license stock images is nearly here. Even in its current state, the ease and simplicity of the DALL-E API make this the perfect candidate for consuming in a CMS.

With the fresh release of the DALL-E API, I couldn't think of a better time to give it a whirl and integrate it into Sitecore. 

🥁 Introducing...the ✨ DALL-E OpenAI Image Generator for Sitecore ✨ :




The Details

Naturally, Sitecore PowerShell Extensions was the way to go. The 'Context Menu' and 'Insert Item' integration points allow users to open the prompt while targetting any 'Media Folder' - either by right-clicking, expanding Insert, and selecting 'DALL-E OpenAI Image Generator' or selecting 'DALL-E OpenAI Image Generator' from the Folder tab of the active Media Folder.
Both integration points are configured out-of-the-box to only show on items where the Template is a 'Media Folder.'
Once the prompt is toggled, the user can configure their call to the DALL-E API using the following inputs:
  1. A prompt for the DALL-E to interpret.
  2. Image sizing options - restricted to these three sizes at this time: '256x256', '512x512', 'and 1024x1024' (see https://beta.openai.com/docs/api-reference/images/create#images/create-size)
  3. The number of images to generate (DALL-E API allows an integer from 0 to 10. The module limits generation to up to 5)
  4. The base name for new Media Items created in the Media Library after images are generated from DALL-E.



The module also comes bundled with a Settings item located in the following location:
  • /sitecore/system/Modules/PowerShell/Script Library/DALL-E OpenAI Image Generator/OpenAI API Settings
  • {F102AE0D-6A5B-499B-9500-505D0E6F686F}
This item allows users to change or override some default settings - the most important being the 'OpenAI API Key' field, where users must populate their unique API key. after installing the module, You can generate an OpenAI API Key by logging in and navigating to the API Keys settings for your account: https://beta.openai.com/account/api-keys
The Setting item's Template is located in the following location:
  • /sitecore/templates/Modules/DALL-E OpenAI Image Generator
  • {169EE16E-5D44-4CC1-B475-A2DFC8B08A0B}

The Code

The complete source code is available on GitHub, along with the Sitecore package (see https://github.com/strezag/DALL-E-OpenAI-Image-Generator-for-Sitecore/releases) and installation instructions: 
https://github.com/strezag/DALL-E-OpenAI-Image-Generator-for-Sitecore

The module is totally open to any community contributions or general suggestions. Beyond that, feel free to fork a version of your own a build on top of it.

Final Thoughts

I'm really excited about this space and where I believe it's headed. As this technology continues to grow, we should expect an expansion of the DALL-E API (and others) that will almost certainly redefine how we work with digital visual assets.

Enjoy! 🤖

0 comments:

Post a Comment