# Review RAG Indexer

This code is responsible for creating indexes for the Review RAG service. If
a compatible index exists from a previous run, it will be incrementally built
upon. Otherwise, a fresh index will be created.

## Process Overview

The following steps are the general blocks of work that the script performs in
order to create an index.

<!-- TODO(b/517156708): Add additional sections as the script is completed. -->

### Determine Run Type

The script looks for the previous run's manifest information in CIPD. If the
manifest does not exist (indicating no previous runs have succeeded) or the
script determines that it is no longer relevant (e.g. the window over which
the index is created does not match), then the run will be marked as a clobber
run with a fresh index created. Otherwise, only new data since the last run
will be processed.

### Process Local Git Data

All git commits in the determined window are examined and the following actions
are performed:

  1. Relevant information from the CL is pulled and stored in a more
     script-compatible format such as CL description, revision, and commit
     position.
  2. The state of `DIR_METADATA` and related files are reconstructed for the
     first commit and incrementally updated when a commit changes relevant
     files. The correct `DIR_METADATA` state is then associated with each local
     commit that gets processed.