curated_feed.score
==================

.. py:module:: curated_feed.score

.. autoapi-nested-parse::

   Turning a day of candidates into the model's judgements.

   A scorer produces the raw response text and nothing else. Persisting it,
   validating it and retrying live here, in one place, so that every backend gets
   the same treatment and a new one is a single small class.

   Two backends so far, neither needing an API client or a key:

   - `StubScorer` invents a response from publication dates alone. It exists to
     prove the stages after it — selection, rendering, publishing — against a real
     feed before a token is spent.
   - `FileScorer` reads a response saved from a chat session, which is how the
     evaluation workflow drives the real pipeline.

   The failure policy is decided here rather than at six in the morning: save the
   raw text before parsing, retry once with the validation error appended, and on a
   second failure publish nothing. The previously published feed stays valid and
   served, so a failed day costs one missing day.



Attributes
----------

.. autoapisummary::

   curated_feed.score.CODE_FENCE
   curated_feed.score.NO_SUBSTANCE
   curated_feed.score.MAX_ATTEMPTS
   curated_feed.score.RESPONSE_STAGE


Exceptions
----------

.. autoapisummary::

   curated_feed.score.ScoreError


Classes
-------

.. autoapisummary::

   curated_feed.score.Scorer
   curated_feed.score.StubScorer
   curated_feed.score.FileScorer


Functions
---------

.. autoapisummary::

   curated_feed.score.score_day
   curated_feed.score.parse_response
   curated_feed.score.read_response
   curated_feed.score.unfence
   curated_feed.score.make_scorer
   curated_feed.score.format_summary
   curated_feed.score.build_parser
   curated_feed.score.main


Module Contents
---------------

.. py:data:: CODE_FENCE

.. py:data:: NO_SUBSTANCE
   :value: 'no-substance'


.. py:data:: MAX_ATTEMPTS
   :value: 2


.. py:data:: RESPONSE_STAGE
   :value: 'response'


.. py:exception:: ScoreError

   Bases: :py:obj:`Exception`


   Raised when a usable response cannot be obtained.


.. py:class:: Scorer

   Bases: :py:obj:`Protocol`


   Anything that can produce a response for a day of candidates.

   `feedback` carries the validation error from a previous attempt, and is
   `None` on the first. A backend that cannot act on it should say so rather
   than silently repeating itself.


   .. py:attribute:: name
      :type:  str


   .. py:method:: generate(candidates: collections.abc.Sequence[curated_feed.models.Candidate], prompt: curated_feed.policy.Prompt, *, feedback: str | None = None) -> str


.. py:class:: StubScorer(*, items: int)

   Grades the most recently published candidates, calling nothing.

   Judgement is not the point: publishing something Feedly can ingest is. The
   graded items therefore clear the floor on consequence alone, and are marked
   durable so the staleness adjustment cannot pull them back under it.


   .. py:attribute:: name
      :value: 'stub'



   .. py:attribute:: items


   .. py:method:: generate(candidates: collections.abc.Sequence[curated_feed.models.Candidate], prompt: curated_feed.policy.Prompt, *, feedback: str | None = None) -> str


.. py:class:: FileScorer(path: pathlib.Path)

   Reads a response saved from a chat session.

   This is what connects the manual evaluation loop to the pipeline: export a
   day, paste it into a chat, save the JSON, and the real selection, rendering
   and publishing run against it.


   .. py:attribute:: name
      :value: 'file'



   .. py:attribute:: path


   .. py:method:: generate(candidates: collections.abc.Sequence[curated_feed.models.Candidate], prompt: curated_feed.policy.Prompt, *, feedback: str | None = None) -> str


.. py:function:: score_day(scorer: Scorer, candidates: collections.abc.Sequence[curated_feed.models.Candidate], prompt: curated_feed.policy.Prompt, *, raw_path: pathlib.Path) -> curated_feed.models.Response

   Obtain a validated response for one day.

   The raw text of each attempt is written to `raw_path` before it is parsed, so
   a response that fails validation can still be read afterwards — fence and
   all, since what is saved is what arrived. The last attempt is the one left
   on disk.


.. py:function:: parse_response(raw: str) -> curated_feed.models.Response

   Validate raw response text, tolerating a Markdown code fence.


.. py:function:: read_response(path: pathlib.Path) -> curated_feed.models.Response

   Read a saved response artifact.

   The artifact is the raw text the model returned, fence and all, because it
   is the audit trail for a day that went wrong. Everything that reads it back
   therefore has to be as tolerant as the parse that first accepted it — which
   is why this exists rather than a bare `read_artifact(path, Response)`.


.. py:function:: unfence(text: str) -> str

   The JSON inside a Markdown code fence, or the text unchanged.

   Asking for bare JSON does not reliably get it: wrapping it in a fence is a
   deeply trained habit, and a response saved from a chat session almost always
   arrives fenced. Spending the day's one retry on punctuation would be absurd,
   so the fence is simply removed here.

   >>> unfence('```json\n{"graded": []}\n```')
   '{"graded": []}'
   >>> unfence('{"graded": []}')
   '{"graded": []}'


.. py:function:: make_scorer(config: curated_feed.config.Config, *, response_path: pathlib.Path | None) -> Scorer

   The scorer named in configuration.

   The provider SDK is imported only when it is the one selected, so the
   offline backends — and every other command — stay free of it.


.. py:function:: format_summary(response: curated_feed.models.Response, *, candidates: int, path: pathlib.Path, scorer: Scorer) -> str

   Human-readable run report.


.. py:function:: build_parser() -> argparse.ArgumentParser

.. py:function:: main(argv: collections.abc.Sequence[str] | None = None) -> int