AI can make software dramatically more capable.
It can also introduce a different class of risk.
AI features can produce variable outputs, behave differently as context changes, inherit limitations from third-party models and data, and fail in ways that are difficult to predict using traditional testing alone.
That is why LiteByte uses the NIST Artificial Intelligence Risk Management Framework (AI RMF 1.0) as a reference framework for the way we design, integrate, test, deploy and monitor material AI functionality.
It gives us a structured way to ask four questions throughout an AI system's lifecycle:
How is it governed?
What risks exist in this particular context?
How do we measure whether those risks are adequately controlled?
What do we do with the remaining risk?
We govern AI before we build it
Responsible AI starts before the first API call is written.
For material AI functionality, we establish who is responsible for the system, what it is intended to do, what level of risk is acceptable and what controls will be required before production deployment.
We maintain oversight of the AI systems and providers we use and establish clear ownership for the decisions surrounding them.
The objective is simple:
AI risk should have an owner.
It should not disappear somewhere between the client, developer and model provider.
We map the context before deciding AI is the answer
Not every problem needs AI.
Before significant AI functionality is developed, we assess the proposed use case and document:
- why AI is appropriate;
- what the system is intended to do;
- what it must not be used for;
- who will use it;
- who else could be affected by its outputs;
- what information it will process;
- which external AI providers or models it depends on;
- the expected benefits;
- foreseeable harms and failure modes;
- relevant legal, contractual and data-protection considerations;
- required human oversight; and
- the acceptable level of residual risk.
That assessment produces an initial decision to proceed, proceed with additional controls, reassess the approach, or not use AI at all.
AI is a technical option, not an objective in itself.
We treat third-party models as dependencies, not magic
LiteByte will often integrate established foundation models through APIs rather than train a model from scratch.
That changes what we control, but it does not remove our responsibility for the system we deploy.
The model provider may control the underlying architecture, training process, infrastructure and some aspects of model behaviour.
LiteByte controls the application around it.
That can include:
- what information is sent to the model;
- the instructions and context it receives;
- retrieval and knowledge sources;
- what tools and functions it is permitted to use;
- authentication and permissions;
- output validation;
- human approval;
- logging;
- monitoring;
- fallback behaviour; and
- how the complete system is tested.
We therefore assess third-party AI providers as part of the architecture and consider their data-processing arrangements, known limitations, availability, security, model changes and the impact of replacing or losing that dependency.
Using a reputable AI provider is useful.
It is not a substitute for engineering the surrounding system responsibly.
We define what trustworthy means for the use case
NIST identifies seven characteristics associated with trustworthy AI.
We consider each characteristic and decide what it means for the product we are building.
Valid & Reliable
Does the AI perform its intended task well enough to be depended upon?
We define how the AI should be evaluated for the particular use case.
Depending on the system, that might include factual accuracy, completion success, false positives and negatives, structured-output correctness, robustness, consistency or human-rated output quality.
A model being generally capable is not evidence that it is reliable for a specific application.
Safe
What happens when the AI gets something wrong?
Where AI failure could create meaningful harm, we design safeguards around it.
That can include restricted functionality, human approval, output constraints, safe fallback behaviour, escalation rules and the ability to disable AI functionality when necessary.
The more serious the possible consequence, the stronger the control and evidence we require.
Secure & Resilient
Can the AI functionality be manipulated, abused or made to exceed its authority?
AI introduces attack surfaces that do not exist in ordinary application logic.
Where relevant, we test areas such as prompt injection, malicious input, inappropriate tool use, permission boundaries, information leakage and failure of third-party AI services.
AI does not receive unlimited access to a system simply because doing so is convenient.
Tools, data and actions are constrained according to the principle of least privilege.
Accountable & Transparent
Can we understand what AI is doing, and who is responsible for it?
Where appropriate, users are informed that AI is involved.
Material AI functionality has an identified owner, significant system behaviour can be traced through appropriate logs, and the provider/model and surrounding controls are documented.
AI does not remove human accountability for the product.
Explainable & Interpretable
Does the person relying on the AI have enough information to use its output responsibly?
Different products require different levels of explanation.
For one system, it may be sufficient to clearly identify an AI-generated suggestion.
Another may need supporting sources, reasons for a recommendation, confidence information or a clear explanation of the limitations of the output.
We decide what level of explanation is appropriate based on the consequence of relying on the AI.
Privacy-Enhanced
Are we sending the AI only the information it actually needs?
We consider privacy during architecture and implementation rather than after integration.
That includes data minimisation, provider processing terms, retention, access controls, logging, confidential information and whether personal or sensitive data should be removed or transformed before reaching an external AI service.
The easiest piece of sensitive information to protect is often the information that never needed to be sent in the first place.
Fair, with harmful bias managed
Could the AI create materially different or harmful outcomes for different people or groups?
Where the use case could affect people differently, we assess that risk explicitly.
That can require representative test scenarios, examination of outputs across relevant groups, specialist input and additional human oversight.
Where fairness has little meaningful application to a particular feature, we record why rather than pretending every characteristic carries the same risk in every system.
We test the complete AI system
We do not consider an AI feature tested because the underlying model provider has tested its model.
Before deployment, LiteByte evaluates the system as it will actually be used.
That may include:
- normal user scenarios;
- edge cases;
- ambiguous inputs;
- factuality and hallucination testing;
- adversarial inputs;
- prompt injection;
- attempts to bypass instructions;
- privacy and information leakage;
- tool and function permissions;
- harmful or inappropriate outputs;
- bias and fairness;
- provider/API failure;
- rate limits and timeouts;
- malformed AI responses;
- human-review workflows;
- fallback behaviour; and
- performance under realistic conditions.
Test scenarios, evaluation methods, expected thresholds and results are documented.
Where appropriate, testing uses representative examples of the environment in which the system will actually operate rather than relying entirely on artificial demonstration cases.
Higher-risk AI gets independent review
The level of assurance should reflect the level of potential harm.
For AI classified by LiteByte as High Risk, production approval requires review by a competent team member who was not directly responsible for developing the AI functionality.
That reviewer considers the intended purpose, identified risks, controls, evaluation evidence, remaining limitations, residual risk and production-monitoring arrangements before approving deployment.
Where the subject matter demands specialist expertise (for example healthcare, regulated financial decisions, legal consequences or significant safety risks) additional domain, legal, security or other specialist review may also be required.
The person who built something should not always be the only person deciding that it is safe to ship.
Deployment is a decision, not the end of testing
Once development and evaluation are complete, we make an explicit production decision.
For material AI systems we document:
- whether the intended purpose has been achieved;
- whether required controls are operating;
- test and evaluation results;
- known limitations;
- unresolved findings;
- residual risks;
- who has accepted those risks;
- production-monitoring requirements; and
- any conditions attached to deployment.
A significant risk that cannot reasonably be controlled is a reason to change the design, or not deploy the AI.
We continue managing AI after launch
AI risk does not stop when the feature reaches production.
Depending on the system, ongoing monitoring may include:
- model/API failures;
- output quality;
- unexpected AI behaviour;
- user complaints;
- human overrides;
- security events;
- provider changes;
- model deprecation;
- latency and availability;
- unexpected data exposure; and
- other indicators relevant to the particular product.
Material changes to the provider, model, purpose, data, level of autonomy or deployment context can trigger reassessment and regression testing.
That matters particularly when using third-party models: application code can remain unchanged while the behaviour of an external AI dependency changes.
We plan for failure
Every significant AI integration needs an answer to:
What happens if the AI stops being trustworthy tomorrow?
Where proportionate, systems are designed so that AI functionality can be disabled, overridden, replaced or moved to a safe/manual fallback.
We establish how incidents are identified, escalated, investigated and recovered from, and who needs to be informed.
When AI functionality is eventually retired, access can be revoked, automated processes stopped, relevant data handled appropriately and the system removed from our AI inventory.
Responsible deployment includes responsible exit.
We give users a route back to a human
Where AI materially affects a person, appropriate mechanisms should exist for reporting problems and obtaining human involvement.
Depending on the product, that can include:
- reporting incorrect AI output;
- reporting harmful or inappropriate behaviour;
- requesting human review;
- challenging an AI-supported outcome; and
- feeding real-world failures back into ongoing evaluation.
Feedback from production is evidence.
It should inform how the AI system is tested and improved over time.
Our AI lifecycle
We apply the NIST AI RMF through four connected activities:
GOVERN
Set the rules and accountability.
Establish ownership, policy, risk tolerance, responsibilities and oversight.
MAP
Understand the specific system and its risks.
Define its purpose, context, users, affected parties, data, dependencies, limitations, benefits and potential harms.
MEASURE
Prove what we can.
Test the system, measure relevant trustworthiness characteristics, document limitations and evaluate whether the controls actually work.
MANAGE
Make decisions based on the evidence.
Prioritise risks, implement controls, decide whether to deploy, monitor the system, respond to incidents and continually improve it.
Then repeat as the system and its environment change.
What alignment means at LiteByte
For us, alignment with the NIST AI Risk Management Framework means AI risk management is built into the engineering lifecycle rather than added as a statement at the end.
For material AI systems we aim to be able to answer five questions clearly:
Why are we using AI here?
What could reasonably go wrong?
What have we done to reduce those risks?
What evidence do we have that the system performs acceptably?
What happens if it stops doing so after launch?
If those questions cannot be answered, the system is not ready simply because the API works.