curated_feed.models =================== .. py:module:: curated_feed.models .. autoapi-nested-parse:: Data contracts for the curation pipeline. Three of these are boundaries rather than conveniences. `Candidate` is the corpus record schema, consumed by later stages and by manual evaluation sessions. `Response` is what comes back from the model, which is untrusted input and is validated as such. `SelectionResult` is the record of what was chosen and why. Validation here covers shape and range only. Conflicts *between* records — the same article both dropped and listed as a duplicate, or a candidate the response never mentions — are resolved during selection, where they can be recorded and reported rather than merely rejected. Attributes ---------- .. autoapisummary:: curated_feed.models.MIN_CONSEQUENCE curated_feed.models.MAX_CONSEQUENCE curated_feed.models.TIMESTAMP_FORMAT curated_feed.models.VERDICT_FREE curated_feed.models.VERDICT_PAYWALLED curated_feed.models.VERDICT_UNKNOWN Classes ------- .. autoapisummary:: curated_feed.models.Candidate curated_feed.models.Graded curated_feed.models.Response curated_feed.models.Adjustment curated_feed.models.Selection curated_feed.models.SelectionResult curated_feed.models.Verdict curated_feed.models.PublishedEntry curated_feed.models.FeedMeta curated_feed.models.DeployMeta Module Contents --------------- .. py:data:: MIN_CONSEQUENCE :value: 1 .. py:data:: MAX_CONSEQUENCE :value: 3 .. py:data:: TIMESTAMP_FORMAT :value: '%Y-%m-%dT%H:%M:%SZ' .. py:data:: VERDICT_FREE :value: 'free' .. py:data:: VERDICT_PAYWALLED :value: 'paywalled' .. py:data:: VERDICT_UNKNOWN :value: 'unknown' .. py:class:: Candidate(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` One normalised feed item. The field order below *is* the corpus record schema, and JSON is written in it. Changing the fields, their order or their meaning is a deliberate act. .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:attribute:: id :type: str .. py:attribute:: url :type: str .. py:attribute:: title :type: str .. py:attribute:: summary :type: str .. py:attribute:: source :type: str .. py:attribute:: source_url :type: str .. py:attribute:: folder :type: str | None .. py:attribute:: published_at :type: datetime.datetime | None .. py:attribute:: published_missing :type: bool .. py:attribute:: fetched_at :type: datetime.datetime .. py:class:: Graded(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` The model's reading of one article, as specified in `policy/prompt.md`. Every field is a judgement that requires having read the article. Nothing here is arithmetic, and nothing here knows the floor, the item cap or the ordering — those belong to selection. .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:attribute:: index :type: int :value: None .. py:attribute:: consequence :type: int :value: None .. py:attribute:: topics :type: list[str] :value: None .. py:attribute:: penalties :type: list[str] :value: None .. py:attribute:: durable :type: bool :value: False .. py:attribute:: duplicates :type: list[int] :value: None .. py:attribute:: independent_sources :type: int :value: None .. py:attribute:: discretionary :type: bool :value: False .. py:attribute:: discretionary_reason :type: str | None :value: None .. py:class:: Response(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` A day's judgements: what was dropped, and what was graded. .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:attribute:: dropped :type: dict[str, list[int]] :value: None .. py:attribute:: graded :type: list[Graded] :value: None .. py:property:: accounted_indices :type: dict[int, int] How often the response mentions each candidate number. The coverage invariant wants exactly one mention per candidate. Counting rather than testing membership is what makes both failures visible: a count of zero is an article never considered, and a count above one is an article claimed twice. .. py:class:: Adjustment(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` One weighted term of a score, kept beside the reason it was applied. .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:attribute:: reason :type: str .. py:attribute:: value :type: float .. py:class:: Selection(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` A selected article, with the arithmetic that put it there. `rationale` is the only field published; everything else exists for tuning and stays in the private selection log. .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:attribute:: candidate_id :type: str .. py:attribute:: rank :type: int :value: None .. py:attribute:: consequence :type: int .. py:attribute:: preference :type: int .. py:attribute:: topics :type: list[str] :value: None .. py:attribute:: adjustments :type: list[Adjustment] :value: None .. py:attribute:: score :type: float .. py:attribute:: discretionary :type: bool :value: False .. py:attribute:: discretionary_reason :type: str | None :value: None .. py:attribute:: rationale :type: str :value: '' .. py:attribute:: duplicates :type: list[str] :value: None .. py:class:: SelectionResult(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` The `state/selections/DATE.json` artifact. Rejects are grouped by reason code rather than listed one by one, which keeps the artifact small while preserving a complete account of the day. `anomalies` records what the response got wrong and selection had to resolve; an empty list is the normal case, and a growing one is a policy problem. .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:attribute:: run_date :type: datetime.date .. py:attribute:: candidate_count :type: int :value: None .. py:attribute:: selected :type: list[Selection] :value: None .. py:attribute:: rejected :type: dict[str, list[str]] :value: None .. py:attribute:: anomalies :type: list[str] :value: None .. py:class:: Verdict(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` One line of `state/paywall.jsonl`: whether one article can be read. Keyed by candidate id and written once. The log is append-only and outlives a run, so `detail` carries why a verdict was reached — "HTTP 403", "no markup" — which is the whole audit trail for a day whose feed looks short. .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:attribute:: candidate_id :type: str .. py:attribute:: url :type: str .. py:attribute:: verdict :type: Literal['free', 'paywalled', 'unknown'] .. py:attribute:: checked_at :type: datetime.datetime .. py:attribute:: detail :type: str :value: '' .. py:class:: PublishedEntry(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` One line of `state/published.jsonl`, and one entry of the feed. Denormalised on purpose: rendering is then a function of this log alone, and cannot be changed by a later edit to the corpus or the selection. `summary` is the source's own description, carried here for the same reason everything else is. It defaults to empty because the log is append-only and outlives its schema: lines written before the field existed have to stay readable, and they render as they always did. .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:attribute:: candidate_id :type: str .. py:attribute:: url :type: str .. py:attribute:: title :type: str .. py:attribute:: summary :type: str :value: '' .. py:attribute:: source :type: str .. py:attribute:: published_at :type: datetime.datetime | None .. py:attribute:: rationale :type: str .. py:attribute:: run_date :type: datetime.date .. py:class:: FeedMeta(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` What the last render produced, so the next one can tell if it changed. The feed-level `` moves only when the content hash does. Stamping the run time on every render makes a reader treat every item as modified. .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:attribute:: content_hash :type: str .. py:attribute:: updated :type: datetime.datetime .. py:class:: DeployMeta(/, **data: Any) Bases: :py:obj:`pydantic.BaseModel` What the last upload sent, so an unchanged feed is not sent again. Written only after a real upload succeeds. A failed one leaves the previous hash in place, which is what makes the next run retry rather than assume. .. py:attribute:: model_config Configuration for the model, should be a dictionary conforming to [`ConfigDict`][pydantic.config.ConfigDict]. .. py:attribute:: content_hash :type: str .. py:attribute:: deployed_at :type: datetime.datetime