OpenAI on Tuesday announced that the company is integrating its latest image-generation technology into its chatbot, ChatGPT, using the GPT-40 model the company unveiled last year.
With the combined technology, ChatGPT could become a one-stop shop for artificially generated content, replacing Dall-E 3, a separate system just for image creation.
Instead of Dall-E 3, ChatGPT’s image generation will instead use Sora, a separate platform the company unveiled last year that is dedicated to video generation. OpenAI didn’t detail any plans for video-generation capabilities using ChatGPT in its press release.
OpenAI said the biggest advancement in the chatbot’s image ability is in the quality of its text rendering abilities. The company claims its systems can now generate images with text that is meaningful and readable — not warped and typo-ridden.
“We trained our models on the joint distribution of online images and text, learning not just how images relate to language, but how they relate to each other,” the company said in a press release. “Combined with aggressive post-training, the resulting model has surprising visual fluency, capable of generating images that are useful, consistent, and context-aware.”
The company said this system will be able to use user-uploaded images as visual inspirations and is better at following instructions. OpenAI claims that “while other systems struggle with ~5-8 objects, GPT‑4o can handle up to 10-20 different objects.”
The advancements come with some trade-offs, however: The model might crop longer images near the bottom, is still prone to seeing things that aren’t there, and struggles to render non-Latin languages or images that contain text at a very small size.
GPT-4o is available immediately on ChatGPT’s plus, pro, team, and free subscription tiers, but “plus” subscribers have higher usage limits than “free” subscribers. The company said the feature will roll out soon for enterprise and edu users, as well as for developers using the API.
Here are some of the image demonstrations from the company’s press release.
