OpenAI

View on Github

This OpenAI app in Blackbird grants you access to all API endpoints and models that OpenAI has to offer from completion, chat, edit to DALL-E image generation and Whisper.

Before setting up

Before you can connect you need to make sure that:

• You have an OpenAI account.
• You have generated a new API key in the API keys section, granting programmatic access to OpenAI models on a ‘pay-as-you-go’ basis. With this, you only pay for your actual usage, which starts at $0,002 per 1,000 tokens for the fastest chat model. Note that the ChatGPT Plus subscription plan is not applicable for this; it only provides access to the limited web interface at chat.openai.com and doesn’t include OpenAI API access. Ensure you copy the entire API key, displayed once upon creation, rather than an abbreviated version. The API key has the shape sk-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
• Your API account has a payment method and a positive balance, with a minimum of $5. You can set this up in the Billing settings section.

Note: Blackbird by default uses the latest models in its actions. If your subscription does not support these models then you have to add the models you can use in every Blackbird action.

Connecting

1. Navigate to apps and search for OpenAI.
2. Click Add Connection.
3. Name your connection for future reference e.g. ‘My OpenAI connection’.
4. Fill in your API key obtained earlier.
5. Click Connect.

Actions

All textual actions have the following optional input values in order to modify the generated response:

• Model (defaults to the latest)
• Maximum tokens
• Temperature
• top_p
• Presence penalty
• Frequency penalty

For more in-depth information about most actions consult the OpenAI API reference.

Different actions support various models that are appropriate for the given task (e.g. gpt-4 model for Chat action). Action groups and the corresponding models recommended for them are shown in the table below.

Action group	Latest models	Default model (when Model ID input parameter is unspecified)	Deprecated models
Chat	gpt-4o, gpt-o1, gpt-o1 mini, gpt-4-turbo-preview and dated model releases, gpt-4 and dated model releases, gpt-4-vision-preview, gpt-4-32k and dated model releases, gpt-3.5-turbo and dated model releases, gpt-3.5-turbo-16k and dated model releases, fine-tuned versions of gpt-3.5-turbo	gpt-4-turbo-preview; gpt-4-vision-preview for Chat with image action	gpt-3.5-turbo-0613, gpt-3.5-turbo-16k-0613, gpt-3.5-turbo-0301, gpt-4-0314, gpt-4-32k-0314
Audiovisual	Only whisper-1 is supported for transcriptions and translations. tts-1 and tts-1-hd are supported for speech creation.	tts-1-hd for Create speech action	-
Images	dall-e-2, dall-e-3	dall-e-3	-
Embeddings	text-embedding-ada-002	text-embedding-ada-002	text-similarity-ada-001, text-similarity-babbage-001, text-similarity-curie-001, text-similarity-davinci-001, text-search-ada-doc-001, text-search-ada-query-001, text-search-babbage-doc-001, text-search-babbage-query-001, text-search-curie-doc-001, text-search-curie-query-001, text-search-davinci-doc-001, text-search-davinci-query-001, code-search-ada-code-001, code-search-ada-text-001, code-search-babbage-code-001, code-search-babbage-text-001

You can refer to the Models documentation to find information about available models and the differences between them.

Some actions that are offered are pre-engineered on top of OpenAI. This means that they extend OpenAI’s endpoints with additional prompt engineering for common language and content operations.

Do you have a cool use case that we can turn into an action? Let us know!

Chat

• Chat given a chat message, returns a response. You can optionally add a system prompt and/or an image. Also you can add collection of texts and it will be added to the prompt along with the message. Also you can optionally add Glossary as well. Useful if you want to add collection of messages to the prompt.
• Chat with system prompt the same as above except that the system prompt is mandatory.

Assistant

• Message assistant sends a message to a pre-configured assistant. It will execute the assistant and return its message. The assistant action can accept up to 10 files as input. Note: If you want to read the attached files make sure your assistant has Retrieval enabled.

Localization (pre-engineered)

• Post-edit MT given a source segment and NMT translated target segment, responds with a post-edited version of the target segment taking into account typical NMT mistakes.

• Get translation issues given a source segment and NMT translated target segment, highlights potential translation issues. Can be used to prepopulate TMS segment comments.

• Get MQM report performs an LQA Analysis of the translation. The result will be in the MQM framework form. The dimensions are: terminology, accuracy, linguistic conventions, style, locale conventions, audience appropriateness, design and markup. The input consists of the source and translated text. Optionally one can add languages and a description of the target audience.

• Get MQM dimension values uses the same input and prompt as ‘Get MQM report’. However, in this action the scores are returned as individual numbers so that they can be used in decisions. Also returns the proposed translation.

• Translate text given a text and a locale, tries to create a localized version of the text.

• Get localizable content from image retrieves localizable content from given image.

Content repurposing

All actions can take: Target audience, locale, glossary, tone of voice and any additional prompts

• Repurpose content repurposes content to a specific target audience.
• Summarize content creates a summary of content.

Glossary extraction

• Extract glossary extracts a glossary (.tbx) from any (multilingual) content. You can use this in well in conjunction with other apps that take glossaries.

Audiovisual

• Create transcription transcribes the supported audiovisual file formats into a textual response.
• Create English translation same as above but automatically translated into English.
• Create speech generates audio from the text input.

Images

• Generate image use DALL-E to generate an image based on a prompt.

Other

• Create embedding create a vectorized embedding of a text. Useful in combination with vector databases like Pinecone in order to store large sets of data.
• Tokenize text turn a text into tokens. Uses Tiktoken under the hood.

XLIFF operations

All XLIFF actions supports 1.2 and 2.1 versions of the XLIFF format, since these are the most common versions used in the industry, but if you need support for other versions, please reach out to us and we will consider adding support for them.

Currently we are using structured output feature to return the results of the actions. This means that the output of the actions will be structured in a way that is easy to parse and could increase the stability of the actions. However note, Structured Outputs are available in latest large language models, starting with GPT-4o:

• gpt-4o-mini-2024-07-18 and later
• gpt-4o-2024-08-06 and later

You can find more information about structured outputs in the OpenAI documentation.

• Get Quality Scores for XLIFF file Gets segment and file level quality scores for XLIFF files. Optionally, you can add Threshold, New Target State and Condition input parameters to the Blackbird action to change the target state value of segments meeting the desired criteria (all three must be filled).

  Optional inputs:

  • Prompt: Add your criteria for scoring each source-target pair. If none are provided, this is replaced by “accuracy, fluency, consistency, style, grammar and spelling”.
  • Bucket size: Amount of translation units to process in the same request. (See dedicated section)
  • Source and Target languages: By defualt, we get these values from the XLIFF header. You can provide different values, no specific format required.
  • Threshold: value between 0-10.
  • Condition: Criteria to filter segments whose target state will be modified.
  • New Target State: value to update target state to for filtered translation units.

  Output:

  • Average Score: aggregated score of all segment level scores.
  • Updated XLIFF file: segment level score added to extradata attribute & updated target state when instructed.

The Prompt used in this action is “Your input is going to be a group of sentences in source language and their translation into target language. Only provide as output the ID of the sentence and the score number. The score number is a score from 1 to 10 assessing the quality of the translation, considering the following criteria: Criteria provided via Prompt input.”

As an example, the settings in the image below will result in all semegments with a score below or equal to 6 to have their target state updated to “needs-review-translation”.

• Post-edit XLIFF file Updates the targets of XLIFF files

Optional inputs:

• Prompt: Add your linguistic criteria for postediting targets.
• Bucket size: Amount of translation units to process in the same request. (See dedicated section)
• Source and Target languages: By default, we get these values from the XLIFF header. You can provide different values, no specific format required.
• Glossary
• Update locked segments: If true, locked segments will be updated, otherwise they will be skipped. By default, this is set to false.

The Prompt used in this actions is “Your input consists of sentences in source language with their translations into target language. Review and edit the translated target text as necessary to ensure it is a correct and accurate translation of the source text. If you encounter XML tags in the source also include them in the target text, don’t delete or modify them.” By using the “Prompt” optional input, your instructions will be appended to the prompt.

• Process XLIFF file given an XLIFF file, processes each translation unit according to provided instructions in the optinal input “Prompt” (if no Prompt is provided, the source texts will be translated) and updates the target text for each unit.

Whenever a Glossary is added as optional input for any of the above described actions, the Prompt used is “Enhance the target text by incorporating relevant terms from our glossary where applicable. Ensure that the translation aligns with the glossary entries for the respective languages. If a term has variations or synonyms, consider them and choose the most appropriate translation to maintain consistency and precision.”

Here is an example bird for processing XLIFF files:

Bucket size, performance and cost

XLIFF files can contain a lot of segments. Each action takes your segments and sends them to OpenAI for processing. It’s possible that the amount of segments is so high that the prompt exceeds to model’s context window or that the model takes longer than Blackbird actions are allowed to take. This is why we have introduced the bucket size parameter. You can tweak the bucket size parameter to determine how many segments to send to OpenAI at once. This will allow you to split the workload into different OpenAI calls. The trade-off is that the same context prompt needs to be send along with each request (which increases the tokens used). From experiments we have found that a bucket size of 1500 is sufficient for gpt-4o. That’s why 1500 is the default bucket size, however other models may require different bucket sizes.

Eggs

Check downloadable workflow prototypes featuring this app that you can import to your Nests here.

Batch processing

You can use batch (async) actions to process large XLIFF files. The batch action will return a batch object that you can use to check the status of the processing by using Batch ID.

• (Batch) Process XLIFF file - Asynchronously process each translation unit in the XLIFF file according to the provided instructions (by default it just translates the source tags) and updates the target text for each unit.
• (Batch) Post-edit XLIFF file - Asynchronously post-edit the target text of each translation unit in the XLIFF file according to the provided instructions and updates the target text for each unit.
• (Batch) Get Quality Scores for XLIFF file - Asynchronously get quality scores for each translation unit in the XLIFF file.

To get the results of the batch processing, you can use the following actions:

• Get results of the batch process - Get the results of the batch process. This action is suitable only for processing and post-editing XLIFF file and should be called after the batch process is completed.
• Get quality scores results - Get the quality scores results of the batch process. This action is suitable only for getting quality scores for XLIFF file and should be called after the batch process is completed.

Note, that you should specify the correct original XLIFF file in Original XLIFF input. It will help us to construct the correct XLIFF file with updated target segments.

Limitations

• The maximum number of translation units in the XLIFF file is 50 000 because a single batch may include up to 50,000 requests

How to know if the batch process is completed?

You have 3 options here:

1. You can use the On batch finished event trigger to get notified when the batch process is completed. But note, that this is a polling trigger and it will check the status of the batch process based on the interval you set.
2. Use the Delay operator to wait for a certain amount of time before checking the status of the batch process. This is a more straightforward way to check the status of the batch process.
3. Since October 2024, users can rely on Checkpoints to achieve a fully streamlined process. A Checkpoint can pause the workflow until the LLM returns a result or a batch process completes.

We recommend using the On batch finished event trigger with Checkpoints.

Example

This simple example how OpenAI can be used to communicate with the Blackbird Slack app. Whenever the app is mentioned, the message will be send to Chat to generate an answer. We then use Amazon Polly to turn the textual response into a spoken-word resopnse and return it in the same channel.

Missing features

In the future we can add actions for:

• Files
• Moderation
• Fine-tuning

Let us know if you’re interested!

Actions limitations

• For every action maximum allowed timeout are 300 seconds (5 minutes). If the action takes longer than 300 seconds, it will be terminated. Based on our experience, even complex actions should take less than 5 minutes. But if you have a use case that requires more time, let us know.

OpenAI is sometimes prone to errors. If your Flight fails at an OpenAI step, please check https://status.openai.com/history first to see if there is a known incident or error communicated by OpenAI. If there are no known errors or incidents, please feel free to report it to Blackbird Support.

Feedback

Feedback to our implementation of OpenAI is always very welcome. Reach out to us using the established channels or create an issue.