We're excited to share more about Adept Workflow Language (AWL). We've designed AWL to simplify the creation of AI workflows to operate across multiple business applications.

Today, we’re announcing some updates to our strategy and some changes to our leadership and team. More details are in our blog:

We've updated our post on Fuyu-Heavy with some new multimodal eval results. On one of these benchmarks (AI2D) Fuyu-Heavy even outperforms Gemini-Ultra!.

RT @Gradio: Fuyu-Heavy from @AdeptAILabs looks very promising -- this demo shows that the model excels at multimodal reasoning and has a br….

We’re working on further scaling up these models and building useful software agents around them, if that sounds exciting to you, please reach out at

Finally, simply dealing with different image formats and coordinate systems correctly and consistently at training and inference time is itself a big headache.

There was also a serious lack of quality image pre-training data, and so we pulled out a lot of stops to create a good dataset using both existing and synthetically generated data.

There’s a delicate balance between language modeling and image modeling tasks, and we had to develop recipes for striking this balance at scale.

Image models are famously unstable (, and we’ve tweaked the basic Fuyu architecture and our training procedure substantially to stabilize training.

Image data stresses systems – keeping the model fed with data becomes harder, memory usage blows up, and cloud storage bandwidth can become a limiting factor.

Training Fuyu-Heavy wasn’t easy - in addition to the standard hiccups with model scaling, we had to deal with the extra problems associated with training a new architecture on both text and image data….

Here’s a blog post if you want all the details: or read on for highlights.

Introducing Fuyu-Heavy, our new multimodal model. Fuyu-Heavy is the world’s third-most-capable multimodal model, behind only GPT4-V and Gemini Ultra, which are 10-20 times larger. In particular, it outperforms Gemini Pro at both MMLU and MMMU.

Because Workflows is an Experiment, it often requires some careful prompting to make it do what you want. Some sites work better than others. Stay tuned for more Experiments and research updates as we continue to iterate.

Workflows is powered by ACT-2, a multimodal model fine-tuned from the Fuyu family and optimized for UI understanding, knowledge worker data comprehension, and action taking.

You can also build your own Workflows by putting together actions like "click" and "type" through simple English in the Workflow Builder! Perhaps you could automate away ordering your dinner 🍱

Workflows can even teach a novice user to navigate the tool like an expert. Here, a new store manager uses Adept to easily create new discount codes.

Workflows can trivially hop between tools, making moving information between them seamless. An insurance agent can easily extract information from claims emails and insert the data into forums.

Workflows can turn unstructured data into structured data. Here is a workflow that enables Adept to open attached invoices, extract information, and move it into accounts payable software.

Workflows can do repetitive tasks for you. For example, this workflow moves the candidate to a technical interview round once the recruiter is satisfied with their take home test submission.