> ## Documentation Index
> Fetch the complete documentation index at: https://neuraltrust-92b43583-develop.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Connect to NeuralTrust

The TrustTest client provides a powerful interface for persisting and retrieving evaluation artifacts. It allows you to save and load any scenario, test set, and evaluation results through a consistent API.

## Client Implementations

TrustTest offers multiple client implementations:

### NeuralTrustClient

The `NeuralTrustClient` connects to the NeuralTrust API service and provides the following methods:

```python theme={null}
from trusttest.clients import NeuralTrustClient

# Initialize client with your API token and target ID
client = NeuralTrustClient(
    token="your_api_token",
    target_id="your_target_id",
)
```

#### Configuration

The token and target ID are defined in your app settings. You can either:

* Pass them directly when initializing the client
* Set them as environment variables: `NEURALTRUST_TOKEN` and `NEURALTRUST_TARGET_ID`

#### Evaluation Scenarios

```python theme={null}
# Save an evaluation scenario
client.save_evaluation_scenario(evaluation_scenario)

# Load an evaluation scenario by ID
scenario = client.get_evaluation_scenario("scenario_id")
```

#### Test Sets

```python theme={null}
# Save a test set for a specific scenario
client.save_evaluation_scenario_test_set("scenario_id", test_set)

# Update an existing test set or create if it doesn't exist
client.upsert_evaluation_scenario_test_set("scenario_id", test_set)

# Load a test set for a specific scenario
test_set = client.get_evaluation_scenario_test_set("scenario_id")
```

#### Evaluation Results

```python theme={null}
# Save evaluation run results
client.save_evaluation_scenario_run(evaluation_run)

# Load evaluation run results for a scenario
results = client.get_evaluation_scenario_run("scenario_id")
```

#### Evaluators

```python theme={null}
# Save a custom evaluator with optional name and description
client.save_evaluator(evaluator, name="my_evaluator", description="Custom evaluator")

# Load an evaluator by name
evaluator = client.get_evaluator("my_evaluator")
```

#### Remote execution and metrics

After you have saved a scenario (and usually a test set), you can run it on the platform and poll status:

```python theme={null}
run = client.run_evaluation_scenario(
    scenario_id,
    refresh_responses=True,
    notify=False,
)
# or a single case:
# client.run_evaluation_scenario_test_case(scenario_id, test_case_id)

status = client.get_evaluation_run_status(
    scenario_id,
    evaluation_run_id=run.id,
    timeout=10000,
)
test_set_status = client.get_evaluation_test_set_status(scenario_id, timeout=10000)

metrics = client.get_evaluation_scenario_run_metrics(
    scenario_id,
    from_date=None,
    to_date=None,
)
target_metrics = client.get_target_total_metrics()
```

`run_evaluation_scenario` / `run_evaluation_scenario_test_case` return a `RunResult`. Metrics types are `ScenarioMetrics` and `TargetMetrics`.

### FileSystemClient

The `FileSystemClient` stores the same artifacts as JSON under `path` (default `trusttest_db`).

```python theme={null}
from trusttest.clients import FileSystemClient

client = FileSystemClient(path="trusttest_db")
overview = client.get_overview()
# RedTeamingOverview: overall, language[], category[] (by sub_category), frameworks[]
print(overview.overall.percentage_passed)
print(overview.frameworks)
```

`get_overview()` rolls up local red-team results by language, subcategory, and the **framework tags** attached by catalog builders (EU AI Act, OWASP, MITRE ATLAS, ISO/IEC 42001).

## Key Capabilities

All TrustTest clients support these core operations:

* **Evaluation Scenarios**: Save and retrieve evaluation scenario definitions
* **Test Sets**: Manage test sets associated with evaluation scenarios
* **Evaluation Results**: Persist and load evaluation run results
* **Evaluators**: Store custom evaluator configurations

## Example Workflow

Here's how to use the client in a typical evaluation workflow:

```python theme={null}
from trusttest.clients import FileSystemClient
from trusttest.evaluation_scenarios import EvaluationScenario
from trusttest.probes import TestSet

# Initialize a client
client = FileSystemClient()

# Create and save an evaluation scenario
scenario = EvaluationScenario(name="My Test", description="Testing functionality")
client.save_evaluation_scenario(scenario)

# Save a test set for the scenario
test_set = TestSet(test_cases=[...])
client.save_evaluation_scenario_test_set(scenario.id, test_set)

# Later, retrieve the scenario and its test set
loaded_scenario = client.get_evaluation_scenario(scenario.id)
loaded_test_set = client.get_evaluation_scenario_test_set(scenario.id)

# After running an evaluation, save the results
client.save_evaluation_scenario_run(evaluation_run)
```

The client abstraction ensures your evaluation artifacts are consistently stored and retrieved regardless of the underlying storage mechanism you choose.