Playing Valid Chess Moves

Note

To download this example as a Jupyter notebook, click here.

Warning

This example is currently under development (it cannot be used to play a full chess game yet).

In this example, we will use Guardrails to play chess with an LLM and ensure that it makes valid moves.

Objective

We want to generate a valid chess moves for a given board state.

import guardrails as gd
from rich import print

!pip install chess

Collecting chess
  Obtaining dependency information for chess from https://files.pythonhosted.org/packages/d6/d8/15cfcb738d2518daf04d34b23419bd359cbd8e09da50778ebac521774fc8/chess-1.10.0-py3-none-any.whl.metadata
  Downloading chess-1.10.0-py3-none-any.whl.metadata (19 kB)
Downloading chess-1.10.0-py3-none-any.whl (154 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 154.4/154.4 kB 3.0 MB/s eta 0:00:00a 0:00:01
Installing collected packages: chess
Successfully installed chess-1.10.0

Step 1: Create the RAIL Spec

Ordinarily, we would create an RAIL spec in a separate file. For the purposes of this example, we will create the spec in this notebook as a string following the RAIL syntax. For more information on RAIL, see the RAIL documentation. We will also show the same RAIL spec in a code-first format using a Pydantic model.

First we define a custom Validator:

from guardrails.validators import Validator, register_validator, ValidationResult, PassResult, FailResult

from typing import Dict, Any

import chess

BOARD = chess.Board()

@register_validator(name="is-valid-chess-move", data_type="string")
class IsValidChessMove(Validator):

    board = BOARD

    def validate(self, value: Any, metadata: Dict) -> ValidationResult:
        global BOARD
        try:
            # Push the move onto the board.
            BOARD.push_san(value)
        except Exception as e:
            # If the move is invalid, raise an error.
            return FailResult(
                error_message=f"Value {value} is not a valid chess move. {e}"
            )        

        return PassResult()

Then we can define our RAIL spec either as XML:

rail_str = """
<rail version="0.1">
<output>
<string description="A move in standard algebraic notation." format="is-valid-chess-move" name="move" on-fail-is-valid-chess-move="reask" required="true"></string>
</output>
<prompt>
Generate a move for the chess board. The board is currently in the following state:
${board_state}
${gr.complete_json_suffix}
</prompt>
</rail>
"""

Or as a Pydantic model:

from pydantic import BaseModel, Field

prompt = """
Generate a move for the chess board. The board is currently in the following state:
${board_state}
${gr.complete_json_suffix}
"""

class ChessMove(BaseModel):
    move: str = Field(
        description="A move in standard algebraic notation.",
        validators=[IsValidChessMove(on_fail="reask")]
        )

Step 2: Create a Guard object with the RAIL Spec

We create a gd.Guard object that will check, validate and correct the output of the LLM. This object:

1. Enforces the quality criteria specified in the RAIL spec.
2. Takes corrective action when the quality criteria are not met.
3. Compiles the schema and type info from the RAIL spec and adds it to the prompt.

From XML:

guard = gd.Guard.from_rail_string(rail_str)

From a Pydantic model:

guard = gd.Guard.from_pydantic(output_class=ChessMove, prompt=prompt)

We see the prompt that will be sent to the LLM. The {board_state} is substituted with the current state of the board.

print(guard.base_prompt)

Generate a move for the chess board. The board is currently in the following state:
${board_state}

Given below is XML that describes the information to extract from this document and the tags to extract it into.

<output>
    <string name="move" format="is-valid-chess-move" description="A move in standard algebraic notation."/>
</output>


ONLY return a valid JSON object (no other text is necessary), where the key of the field in JSON is the `name` 
attribute of the corresponding XML, and the value is of the type specified by the corresponding XML's tag. The JSON
MUST conform to the XML format, including any types and format requests e.g. requests for lists, objects and 
specific types. Be correct and concise. If you are unsure anywhere, enter `null`.

Here are examples of simple (XML, JSON) pairs that show the expected behavior:
- `<string name='foo' format='two-words lower-case' />` => `{'foo': 'example one'}`
- `<list name='bar'><string format='upper-case' /></list>` => `{"bar": ['STRING ONE', 'STRING TWO', etc.]}`
- `<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" format="1-indexed" 
/></object>` => `{'baz': {'foo': 'Some String', 'index': 1}}`

Let's get the reference to the board.

board = guard.output_schema.move.validators[0].board
board

Step 3: Wrap the LLM API call with Guard

import openai

raw_llm_response, validated_response = guard(
    openai.Completion.create,
    prompt_params={
        "board_state": str(board.move_stack)
        if board.move_stack
        else "Starting position."
    },
    engine="text-davinci-003",
    max_tokens=2048,
    temperature=0.3,
)

Async event loop found, but guard was invoked synchronously.For validator parallelization, please call `validate_async` instead.

The guard wrapper returns the raw_llm_respose (which is a simple string), and the validated and corrected output (which is a dictionary).

We can see that the output is a dictionary with the correct schema and types.

print(validated_response)

{'move': 'e4'}

board

Let's make a move.

board.push_san("e5")
board

Ask for another move from the model.

raw_llm_response, validated_response = guard(
    openai.Completion.create,
    prompt_params={
        "board_state": str(board.move_stack)
        if board.move_stack
        else "Starting position."
    },
    engine="text-davinci-003",
    max_tokens=2048,
    temperature=0.3,
)

Async event loop found, but guard was invoked synchronously.For validator parallelization, please call `validate_async` instead.

board

board.push_san("Nc6")
board