PromptTools#

Welcome to prompttools created by Hegel AI! This repository offers a set of free, open-source tools for testing and experimenting with prompts. The core idea is to enable developers to evaluate prompts using familiar interfaces like code and notebooks.

The prompttools notebook demo.

There are primarily two ways you can use prompttools in your LLM workflow:

  1. Run experiments in notebooks and evaluate the outputs.

  2. Turn evaluations into unit tests and integrate them into your CI/CD workflow via Github Actions.

Please don’t hesitate to star our repo, reach out, and provide feedback on GitHub!

To stay in touch with us about issues and future updates, join the Discord.

Installation#

To install prompttools using pip:

pip install prompttools

To install from source, first clone this GitHub repo to your local machine, then, from the repo, run:

pip install .

You can then proceed to run our examples.

Frequently Asked Questions (FAQs)#

  1. Will this library forward my LLM calls to a server before sending it to OpenAI/Anthropic/etc?

    • No, the source code will be executed on your machine. Any call to LLM APIs will be directly executed from your machine without any forwarding.

Contributing#

We welcome PRs and suggestions! Don’t hesitate to open a PR/issue or to reach out to us via email. Please have a look at our contribution guide and “Help Wanted” issues to get started!

Usage and Feedback#

We will be delighted to work with early adopters to shape our designs. Please reach out to us via email if you’re interested in using this tooling for your project or have any feedback.

License#

We will be gradually releasing more components to the open-source community. The current license can be found in the LICENSE file. If there is any concern, please contact us and we will be happy to work with you.

Module Index#