Troubleshooting Prompts

If a Prompt isn't behaving as expected, this guide will help you identify what's going wrong and how to fix it. Work through the relevant section below, then use Save & Verify to test your changes before saving.

For guidance on writing effective Prompts from the outset, see Common AI Mistakes to Avoid When Writing Craig Prompts.

My Prompt isn't returning any results

Check that the data source is reachable. Open the Execution Trace on the Save & Verify screen and look at the tools Craig called. If a tool shows an error or an unusually long response time, the data source may be temporarily unavailable. Wait a few minutes and run the Prompt again.

Check that your location is recognized. If Craig can't resolve the location in your Prompt, it may not know which data source to call. Make sure the location is named clearly and specifically — include a region, province, state, or country where needed to avoid ambiguity. If your organization uses a local name for a location, add the official place name or coordinates to your Contexts so Craig can resolve it correctly.

Check that the right tool was called. In the Execution Trace, confirm that Craig called the data source you expected. If it called a different tool — or no tool at all — your Prompt may not be specific enough about what data to retrieve. Add more detail about the data type and location.

My Prompt is returning unexpected or incorrect results

Check your examples for real data values. If your Prompt includes an example with realistic numbers — a flow rate, a temperature, a tide height — Craig may be reproducing those values instead of fetching live data. Replace all real values in examples with [[placeholders]]  . See Common AI Mistakes — Mistake #3.

Check for conflicting instructions. If your Prompt contains two rules that say different things about the same thing — for example, two different format instructions — Craig will pick one and apply it everywhere. Review your Prompt for duplicated or contradictory instructions and consolidate them into a single clear rule. See Common AI Mistakes — Mistake #2.

Check that you're asking for values the data source actually provides. If the output contains a calculated value — a trend, a duration, a difference — that doesn't appear directly in the data, Craig may have computed it itself. This can produce inaccurate results, particularly at inflection points like the turn of a tide. Ask only for values the tool returns directly, or ask for the calculation to be added to the tool's output. See Common AI Mistakes — Mistake #8.

Check whether the data source has been updated. Some data sources update infrequently. If your Prompt is returning stale-looking results, check the source URL in the Execution Trace directly to confirm whether the underlying data has changed.

My Prompt is including content it shouldn't

Replace negative instructions with a positive list. Instructions like "do not include X" are among the least reliable ways to guide Craig's output — naming a forbidden item often primes Craig to include it. Replace any "do not include" instructions with a clear list of exactly what to include, and leave everything else out by implication. See Common AI Mistakes — Mistake #4.

Check your conditional logic. If a section is appearing when it shouldn't, the condition controlling it may not be written clearly enough for Craig to evaluate. Craig errs on the side of including content when uncertain. Rephrase the condition as a simple range check using values Craig can read directly, rather than values it has to calculate. See Common AI Mistakes — Mistakes #1 and Mistake #5.

Check your examples for format or label bleed. If unexpected labels, headings, or structure are appearing in the output, Craig may be copying them from an example in your Prompt. Make sure all examples use [[placeholders]]  rather than real labels or values, and that no example contains markdown formatting such as bold or bullet points.

My Prompt is producing inconsistent results

Check your cache duration setting. If your Prompt is configured with a cache duration, Craig will return the same data for the length of that window rather than fetching fresh data on every run. If you're seeing results that don't reflect current conditions, check whether caching is enabled and whether the duration is appropriate for the data type. Rapidly changing data — severe weather alerts, tsunami warnings — should not be cached. See Configuring Prompts.

Check whether you've recently edited the Prompt. Editing a Prompt triggers a new discovery phase on the next run, which may result in a different model being assigned. If results feel different after an edit, check the model icon on your Prompt — it may have changed. See Fast and Thinking: How Craig Chooses the Right Model.

Run the Prompt several times and compare. If results vary between runs without any changes to the Prompt or underlying data, the instructions may be ambiguous enough for Craig to interpret differently each time. Review your Prompt for vague output specifications and replace them with precise, named field lists. See Common AI Mistakes — Mistake #5.

My Prompt has been assigned the Thinking model but I expected Fast

Craig assigns the Thinking model when it determines the task requires conditional logic or multi-step reasoning during the discovery phase. If you expected Fast, review your Prompt for:

• If/then language or conditional rules.
• Instructions that ask Craig to compare, calculate, or evaluate rather than simply retrieve and return.
• Multiple rules covering the same idea in different ways, which can introduce ambiguity that Craig resolves through reasoning.

Simplifying the Prompt — removing conditional logic where it isn't genuinely needed, or splitting a complex Prompt into two simpler ones — may result in Fast being assigned on the next discovery run. See Fast and Thinking: How Craig Chooses the Right Model.

Using the Execution Trace

The Execution Trace is your most useful diagnostic tool. It's available on the Save & Verify screen after running a Prompt and shows exactly what Craig did during execution — which data sources it called, how long each call took, and the URLs it retrieved from.

When troubleshooting, check the Execution Trace for:

• Tools called — did Craig call the data sources you expected? If it called the wrong tool or skipped one entirely, your Prompt may need to be more specific about the data type or location.
• Response times — a tool that took significantly longer than others, or returned an error, may indicate a temporarily unavailable data source. The sample below shows normal response times for comparison.
• Source URLs — confirm that Craig retrieved data from the correct location. If the URL doesn't match what you expected, your location may need to be more precisely specified.

Example Execution Trace:

1. Get the rendered visible text of a JavaScript-heavy webpage by loading it in a headless browser: https://www.cypressmountain.com/mountain-report (20364ms)
2. Get the weather forecast for a Canadian location: Cypress Mountain (55ms)
3. Get active weather warnings and alerts for a Canadian location: Cypress Mountain (94ms)

In this example, Craig called three tools: a web page fetch, a weather forecast, and a weather warnings check — all for the same location. Steps 2 and 3 returned quickly; Step 1 took longer because it loaded a full webpage in a browser. All three completed successfully.

If a tool in your trace shows an error or no result, that's where to focus your attention first.

Still not getting the results you need?

If you've worked through the steps above and your Prompt still isn't behaving as expected, select Report an Issue and provide a description of what you expected versus what you received.