Image Knowledge Base

Large language models are powerful content machines, but they lack when it comes to generating fully fledged articles with visual aid as a human writer would.

Moonlit's "Image Knowledge Base" feature empowered these AI models to bypass this limitation.

This is done by using state-of-the-art models for accurately attaching a description to each image, the description is then vectorized so that we can later fetch the top X most relevant descriptions to a specified query along with the associated image url. We can then pass these relevant image with their descriptions to a language model and instruct it to use the images in the content it generates.

In this guide, we will walkthrough how you can start utilizing this powerful feature in your content workflows and take your content automation game to the next level.

1. Adding Images

Navigate to your project's Image Knowledge Base section, and click the Add button. This will open up a modal with a form where you can upload your images (you can add up to 10 at once).

Add Images Modal

In addition to the upload box, you'll see a few extra options:

The first checkbox, when ticked will use the AI to generate a name for the image, otherwise the original upload file name will be used. We recommend this is always ticked unless the image is slightly abstract and the file name accurately describes within the proper context.

Context: As the field description suggests, this helps the AI understand what these images relate to. In the image example, we've set the context to "I run an outdoors equipement ecommerce store" so if I upload a picture of a forest, the AI model is more likely to associate it with hiking, outdoors, and camping. The more vague the images are, the more we'd recommend you be elaborate with the context. You can also tick the 'Save Context' checkbox if you want to add more images later that will use the same context.

Image Knowledge Base

When an Image is uploaded it will go through 3 stage: processing, vectorizing, and completed. When processing the AI will analyze the image and write the detailed description. Then in the vectorizing step, the generated description has embeddings created (Numerical representation of text) so that we can then semantically compare images and fetch only the ones we need in our workflows.

Adding Image Retrieval to Workflows

When an image status is set to "Completed" that means we can start using it in our apps. We created a simple flow for illustration purposes, in which we take an article title as input, pass it through the "Retreive Images" function which will return a list of image urls with descriptions that can relevant to the given query which in this case is our article title. We then passed the results of this function to the Large Language Model (LLM) to write the article with images using the following prompt:

Your task is to write an article with the title "{{Post Title}}".
I want you to also use images throughout your article so I will now supply you with a list of relevant images with their description and urls.
-----
# Images
{{Images}}
-----
The article must be in markdown format. Please proceed with generating the post.
Example App

Results

We ran two examples here, one titled "A guide to hiking equipment" and the latter "A guide to hiking with your dog". The app successfully generated the requested piece with the proper images injected in the right place. Of course, you can be more precise in your prompt, instructing the model how to use the images more precisely, or if you have a content brief/template, you can use that in your prompt as well.

Need more Help?

Please reach out to us through the live chat widget on the bottom right corner or feel free to book a call with us. We're more than excited to explore and help you with your use case!