Safety

We have developed AI guardrails, controls, instructions and policies to ensure that our system is safe, ethical and legal. They act as a safety net to prevent AI from causing harm or being misused. 

These AI guardrails include:

Patient is informed it’s an AI system. The first thing patients will hear is that they are speaking to an AI system. There is some debate in other circles about whether this full disclosure is necessary. But we believe that the ethical thing to do is be transparent. No-one should be tricked into interacting with AI.

Patient can ask to speak to a human. The second thing the patient hears is that they can request to speak to a human. They are informed that this will mean they are placed in queue and it may be some time before a human answers. (There are alternatives: some practices may decide the only alternative is to use their website or app. Out-of-hours the patient could be directed to the out-of-hours service or be told that the practice is closed and they need to call 111 or 999 depending on the seriousness of the condition.)

The system is instructed on emergencies. The system has been instructed that if it detects any of the conditions listed on the 111 website to advise the caller to dial 999. The call is not forwarded directly. The patient does not have to take this advice. They can ask for an appointment for example.

The system will only behave as a GP receptionist. This is the role that the system is instructed to perform. It will not perform any function outside what would be expected of a receptionist.

The system will not diagnose medical conditions. The system will not diagnose any condition even if asked to do so.

The system will not give any medical advice. The system can signpost to services based on the reason for the call but will not give advice itself.

It will not answer inappropriate questions or questions out of scope. Asking general questions (e.g. the capital of France) will get a polite answer informing the caller that the system is a receptionist and cannot answer the question. Similarly for any questions about the system itself, profanity, threats or abuse.

All conversations are recorded. This includes all system responses.

In addition the system never knows the identity of the patient. See Privacy of Patient Information.