NowmasterAI Entity Reference (Governance + Runtime)

This document covers following main entities of NowmasterAI:

• Prompt Templates
• Payload Policies
• Rulesets
• State Mappings
• Record Profiles
• Watermarks
• Artifacts
• Audit Events

It explains purpose, runtime usage, configuration, and operations based on the current implementation.

1) End-to-End Runtime Flow

1. ServiceNow pull fetches changed Incident/Problem/Change/CI rows.
2. State mapping converts source state to canonical phase.
3. Ruleset for (record_type, canonical_phase) returns one or more workflow jobs.
4. Jobs are queued and processed (interactive/background/publish lanes).
5. Worker builds context using:
   • Record Profile (fields, references, journal, attachments mode)
   • Payload Policy (include/exclude minimization for prompt payload)
6. Prompt Template is resolved and sent to LLM.
7. Generated output is stored as Artifact.
8. Publication jobs can write communication/knowledge outputs to u_nowmaster_ai_summary.
9. Governance/runtime actions are appended to Audit Events.
10. Pull checkpoints advance via Watermarks.

2) Entity Matrix

Entity	Table	Main Role
Prompt Templates	nm_prompt_template	Governs prompt body by record type + intent + variant + version
Payload Policies	nm_payload_policy	Governs context field minimization for LLM payload
Rulesets	nm_ruleset	Phase-to-job orchestration
State Mappings	nm_state_mapping	Source state to canonical phase normalization
Record Profiles	nm_record_profile	Persona/use-case context profile
Watermarks	nm_watermark	Source pull checkpointing (backfill + realtime)
Artifacts	nm_artifact	Generated LLM outputs and derived artifacts
Audit Events	nm_audit_event	Governance + runtime audit trail

3) Prompt Templates

Purpose

Defines the exact prompt text and output contract for a workflow intent, e.g. en_incident_complete_summary, de_change_recommendations_service_desk, en_knowledge_article_service_desk.

Where used

• Resolve logic: PromptTemplateService.resolve(...)
• Generation: ArtifactContentGenerator
• Consolidation prompts: ConsolidatedContextService
• Admin management: AdminPromptTemplateController

Key fields

• record_type: target logical template scope (u_nowmaster_ai_summary, knowledge_article, or source type for some workflows)
• intent: workflow intent
• variant: usually persona/profile variant (default fallback)
• output_format: html | json | text
• condition_expr: optional conditional expression
• template_version, priority, active
• template_text: prompt body

Resolution behavior

Search order combines:

• record_type: exact then *
• intent: exact, language-stripped fallback (en_foo -> foo, de_foo -> foo), then *
• variant: exact, default, then *
• active = 1
• sorted by priority desc, template_version desc, updated_at desc
• first template whose condition_expr matches context is selected

Validation rules

• required: name, recordType, intent, variant, outputFormat, templateText
• outputFormat must be html|json|text
• priority range 0..10000
• templateVersion range 1..10000

Configure

• UI: /manage/prompt-templates
• API:
  • GET /api/admin/prompt-templates
  • GET /api/admin/prompt-templates/resolve
  • POST /api/admin/prompt-templates/validate
  • POST /api/admin/prompt-templates/upsert
  • GET /api/admin/prompt-templates/one
  • DELETE /api/admin/prompt-templates?id=... (soft deactivate)

Example upsert

{
  "name": "Incident_Summary_EN",
  "recordType": "u_nowmaster_ai_summary",
  "intent": "en_incident_complete_summary",
  "variant": "default",
  "outputFormat": "html",
  "conditionExpr": "",
  "priority": 95,
  "templateVersion": 1,
  "templateText": "You are an ITSM analyst... ${CONTEXT_JSON}",
  "active": 1
}

4) Payload Policies

Purpose

Controls what source fields are included/excluded in context payload before LLM prompt generation.

Where used

• Resolved in RecordContextService.resolvePayloadPolicy(...)
• Applied in PayloadFieldSelection.select(...)
• Admin management in AdminPayloadPolicyController

Key fields

• record_type, intent, variant
• policy_version, priority, active
• include_fields_csv (required)
• exclude_fields_csv (optional)

Resolution behavior

Search order:

• record_type: exact then *
• intent: exact, language fallback, domain-prefix-stripped fallbacks, then *
  • strips prefixes like incident_, change_request_, problem_, task_, cmdb_ci_, ci_, kb_
• variant: exact, default, then *
• active = 1
• sorted by priority desc, policy_version desc, updated_at desc

Validation rules

• required: recordType, intent, variant
• includeFieldsCsv must have at least one field
• include/exclude overlap is not allowed
• duplicates and empty CSV items are rejected
• priority range 0..10000
• policyVersion range 1..10000

Configure

• UI: /manage/payload-policies
• API:
  • GET /api/admin/payload-policies
  • GET /api/admin/payload-policies/resolve
  • POST /api/admin/payload-policies/validate
  • POST /api/admin/payload-policies/upsert
  • GET /api/admin/payload-policies/one
  • DELETE /api/admin/payload-policies?id=...

Example upsert

{
  "recordType": "incident",
  "intent": "en_incident_complete_summary",
  "variant": "default",
  "policyVersion": 1,
  "priority": 120,
  "includeFieldsCsv": "number,short_description,state,priority,impact,urgency,assignment_group,assigned_to,cmdb_ci",
  "excludeFieldsCsv": "comments,work_notes,comments_and_work_notes",
  "active": 1
}

5) Rulesets

Purpose

Defines which jobs are created when a record is in a canonical phase. This is the orchestration bridge between pull/state and queue/workflows.

Where used

• Pull scheduler calls rulesEngine.planFor(recordType, canonicalPhase)
• Resulting jobType/lane/priority/useCase/personaRole entries are enqueued to job queue
• Admin management in AdminRulesetController

Key fields

• record_type
• canonical_phase
• rules_json with jobs array
• active, updated_at

rules_json formats

• Legacy string entry:
  • "incident.summary.operational"
  • defaults to lane=background, priority=100, useCase=jobType, personaRole=*
• Structured object entry:
  • { "jobType":"incident.summary.complete", "lane":"background", "priority":90, "useCase":"en_incident_complete_summary", "personaRole":"default" }

Validation rules

• rulesJson required and valid JSON
• jobs array required
• each job must be string or object with jobType
• lane must be interactive|background|publish
• priority range 1..1000
• duplicate plan entries rejected

Configure

• UI: /manage/rulesets
• API:
  • GET /api/admin/rulesets
  • GET /api/admin/rulesets/one
  • POST /api/admin/rulesets/validate
  • POST /api/admin/rulesets (upsert)
  • DELETE /api/admin/rulesets?recordType=...&canonicalPhase=...

Example upsert

{
  "recordType": "incident",
  "canonicalPhase": "work_in_progress",
  "rulesJson": {
    "jobs": [
      {
        "jobType": "incident.summary.complete",
        "lane": "background",
        "priority": 90,
        "useCase": "en_incident_complete_summary",
        "personaRole": "default"
      },
      {
        "jobType": "incident.recommendations.service_desk",
        "lane": "background",
        "priority": 80,
        "useCase": "en_incident_recommendations_service_desk",
        "personaRole": "default"
      }
    ]
  }
}

6) State Mappings

Purpose

Normalizes ServiceNow state values/labels into canonical phases used by rulesets (new, assess, in_progress, review, closed, etc.).

Where used

• PhaseMapper.toCanonicalPhase(...) in pull pipeline
• Admin management in AdminStateMappingController

Key fields

• source_system (typically servicenow)
• record_type
• raw_value (normalized key from source state/label)
• raw_label (optional human label)
• canonical_phase
• priority (lower wins)
• active

Runtime behavior

Property nowmaster.state.phaseMode:

• SERVICENOW_STATE: use normalized raw state directly
• CANONICAL: look up nm_state_mapping first; fallback to heuristic mapping if no row exists

Configure

• UI: /manage/state-mappings
• API:
  • GET /api/admin/state-mapping?sourceSystem=...&recordType=...
  • POST /api/admin/state-mapping
  • DELETE /api/admin/state-mapping?sourceSystem=...&recordType=...&rawValue=...

Example upsert

{
  "sourceSystem": "servicenow",
  "recordType": "incident",
  "rawValue": "2",
  "rawLabel": "In Progress",
  "canonicalPhase": "work_in_progress",
  "priority": 10
}

7) Record Profiles

Purpose

Defines persona/use-case aware context profile:

• field scopes
• reference fields
• journal behavior
• attachment policy
• display value mode

This is the main semantic profile contract for context shaping.

Where used

• resolved in RecordContextService.build(...)
• fetch plan parsed by RecordProfileConfig.parseFetchPlan(...)
• applied before canonical mapping and policy reduction

Key fields

• record_type
• use_case
• persona_role
• profile_version
• profile_json
• active

Resolution precedence

Deterministic 8-step fallback:

1. (record_type, use_case, persona_role)
2. (record_type, use_case, *)
3. (record_type, *, persona_role)
4. (record_type, *, *)
5. (*, use_case, persona_role)
6. (*, use_case, *)
7. (*, *, persona_role)
8. (*, *, *)

Then highest profile_version, newest updated_at.

profile_json contract

Required keys:

• record_type
• use_case
• persona_role
• fields (array)
• reference_fields (array)
• journal (array)

Optional keys:

• display_values: value | display | both
• attachments: optional | required | forced
• mapping: object

Attachment mode semantics:

• optional: use attachments if available
• required: expect attachment context; missing attachments can degrade workflow quality
• forced: strongly force attachment processing path in context/profile logic

Configure

• UI: /manage/record-profiles
• API:
  • GET /api/admin/record-profiles
  • GET /api/admin/record-profiles/resolve
  • POST /api/admin/record-profiles/validate
  • POST /api/admin/record-profiles/upsert
  • GET /api/admin/record-profiles/one
  • DELETE /api/admin/record-profiles?id=...
  • helper mapping preview: GET /api/admin/record-profiles/map-main

Example upsert

{
  "recordType": "incident",
  "useCase": "en_incident_complete_summary",
  "personaRole": "default",
  "profileVersion": 1,
  "active": 1,
  "profileJson": {
    "record_type": "incident",
    "use_case": "en_incident_complete_summary",
    "persona_role": "default",
    "display_values": "both",
    "fields": ["sys_id", "number", "short_description", "description", "state", "priority", "impact", "urgency", "assignment_group", "assigned_to", "cmdb_ci", "business_service", "opened_at", "resolved_at", "closed_at"],
    "reference_fields": ["assignment_group", "assigned_to", "cmdb_ci", "business_service"],
    "journal": ["work_notes", "comments"],
    "attachments": "optional"
  }
}

8) Watermarks

Purpose

Stores per-table pull high-water mark so polling is incremental and idempotent.

Two scopes are used:

• base checkpoint: table_name (backfill)
• realtime checkpoint: realtime:<table_name>

Where used

• BackgroundPullScheduler.pullRealtimeLoop() and pullBackfillLoop()
• WatermarkService.getOrInit(...), getOrInitRealtime(...), update(...), updateRealtime(...)

Key fields

• table_name (PK)
• last_sys_updated_on (ServiceNow timestamp string)
• last_sys_id (tie-breaker for equal timestamp rows)
• updated_at

Behavior notes

• Default seed is epoch + zero sys_id.
• Realtime watermark is initialized from base watermark on first use.
• Future-poisoned watermarks are healed automatically using lookback settings.

Relevant properties:

• nowmaster.pull.realtimeLookbackMinutes
• nowmaster.pull.watermarkFutureToleranceMinutes
• nowmaster.pull.backfillSafetyLookbackMinutes
• nowmaster.pull.backfillMaxPages
• nowmaster.pull.backfillMaxRows
• nowmaster.pull.backfillMaxRuntimeMs

Configure

• UI: /manage/watermarks
• API:
  • GET /api/admin/watermarks
  • POST /api/admin/watermarks/{tableName}

Example:

{
  "lastSysUpdatedOn": "2026-03-05 10:04:53",
  "lastSysId": "46f09e75a9fe198100f4ffd8d366d17b"
}

9) Artifacts

Purpose

Immutable generated outputs from LLM workflows and publication steps. They are the main generated evidence layer.

Typical artifact types:

• <record_type>.context
• <record_type>.summary.complete
• <record_type>.recommendations.service_desk
• <record_type>.knowledge.article
• publication artifacts (*.publish)

Where used

• Created by ArtifactService.create(...)
• Read by:
  • admin browser (/api/admin/artifacts/recent)
  • public APIs (/api/artifacts, /api/artifacts/consolidated-context)
• Publication services consume source artifacts and publish to ServiceNow summary table.

Key fields

• context_snapshot_id (foreign key to nm_context_snapshot)
• artifact_type, artifact_schema_version
• content_hash_sha256
• content_inline or content_blob_rel_path
• model_ref, prompt_ref, generation_params_json
• status, created_at, created_by

Public retrieval behavior

GET /api/artifacts returns status:

• ready
• processing
• skipped
• failed
• not_processed

Status is derived from both artifact existence and queue state.

Configure / operate

• UI: /manage/artifacts
• Admin APIs:
  • GET /api/admin/artifacts/recent
  • GET /api/admin/artifacts/one?id=...
• Public APIs:
  • GET /api/artifacts?sourceSystem=...&recordType=...&recordId=...&recordVersionRef=...
  • GET /api/artifacts/consolidated-context?...&jobType=...

10) Audit Events

Purpose

Operational and governance audit trail for actions/events across pipeline and management operations.

Where used

• Written through AuditService.append(...)
• Queried in management dashboard and audit module

Key fields

• ts
• actor_type, actor_id
• action
• entity_type, entity_id
• correlation_id
• details_json
• prev_hash, event_hash (present columns; currently not actively chained in service)

Configure / operate

• UI: /manage/audit
• API:
  • GET /api/admin/audit-events
  • GET /api/admin/audit-events/one?id=...

Filter options:

• action
• correlationId
• entityType + entityId

11) Bootstrap, Defaults, and Import

Bootstrap seeders

If nowmaster.bootstrap.enabled=true, maintenance/seed runners create and reconcile governance defaults:

• prompts
• payload policies
• record profiles
• baseline watermarks
• default rulesets

Governance import

Bulk analyze/apply import from JSON files:

• nm_payload_policy.json
• nm_prompt_template.json
• nm_ruleset.json
• nm_state_mapping.json
• nm_watermark.json (optional by request flags)

12) Configuration Keys That Affect These Entities

Core keys:

• nowmaster.bootstrap.enabled
• nowmaster.state.phaseMode
• nowmaster.pull.*
• nowmaster.workflows.autoEnqueue.*
• nowmaster.servicenow.ai-summary.*
• nowmaster.consolidation.*

Default values source:

• src/main/resources/nm-defaults.yml

Property source precedence:

1. Environment / application.yml
2. nm_properties fallback

13) Operational Guidance (ITIL Context)

• Keep State Mappings aligned with ServiceNow choice values/labels for Incident/Problem/Change.
• Keep Rulesets phase-specific and minimal to avoid over-enqueueing.
• Keep Payload Policies restrictive (least-data principle) to reduce token cost and leakage risk.
• Keep Record Profiles persona/use-case explicit for service desk vs executive vs stakeholder outputs.
• Treat Artifacts as generated evidence and publication source of truth.
• Use Audit Events + correlation_id for end-to-end traceability of Incident/Problem/Change flows.
• Adjust Watermarks only for controlled backfill/recovery operations.

14) Sequence Diagrams

14.1 Pull -> Phase Mapping -> Ruleset -> Queue

sequenceDiagram
  autonumber
  participant Scheduler as BackgroundPullScheduler
  participant SN as SourcePullClient (ServiceNow)
  participant Mapper as PhaseMapper
  participant Rules as RulesEngine
  participant Queue as JobService
  participant WM as WatermarkService

  Scheduler->>SN: fetchChangedRealtime/fetchChangedBackfill(recordType, since)
  SN-->>Scheduler: changed rows (recordId, stateRaw, recordVersionRef)
  loop each changed row
    Scheduler->>Mapper: toCanonicalPhase(recordType, stateRaw)
    Mapper-->>Scheduler: canonicalPhase
    Scheduler->>Rules: planFor(recordType, canonicalPhase)
    Rules-->>Scheduler: [jobType, lane, priority, useCase, personaRole]
    Scheduler->>Queue: enqueueIgnore(...)
  end
  Scheduler->>WM: update/updateRealtime(highwater)

14.2 Context Build -> Profile + Policy Resolution

sequenceDiagram
  autonumber
  participant Worker as JobWorkerEngine
  participant Ctx as RecordContextService
  participant RP as RecordProfileService
  participant PP as PayloadPolicyService
  participant SRC as SourceRecordClient/JournalClient
  participant ATT as RecordAttachmentRepository

  Worker->>Ctx: build(job)
  Ctx->>RP: resolve(recordType,useCase,personaRole)
  RP-->>Ctx: resolved profile (or fallback)
  Ctx->>PP: resolve(recordType,intent,variant)
  PP-->>Ctx: resolved payload policy (or none)
  Ctx->>SRC: fetch main/journal data
  Ctx->>ATT: load attachment refs
  Ctx-->>Worker: payload(meta, main_record, related_records_block, attachments_block)

14.3 Prompt Resolution -> LLM -> Artifact -> Publication

sequenceDiagram
  autonumber
  participant Worker as JobWorkerEngine
  participant Gen as ArtifactContentGenerator
  participant PT as PromptTemplateService
  participant LLM as LlmGateway
  participant Art as ArtifactService
  participant Pub as AiSummaryPublicationService / KnowledgePublicationService
  participant SN as ServiceNowAiSummaryClient
  participant Audit as AuditService

  Worker->>Gen: generate(job, payload)
  Gen->>PT: resolve(recordType,intent,variant,context)
  PT-->>Gen: resolved template
  Gen->>LLM: generate(prompt)
  LLM-->>Gen: model response
  Gen->>Art: create(...artifactType...)
  Art-->>Worker: artifact created
  opt publish workflow
    Worker->>Pub: publish(job)
    Pub->>SN: upsert(request to u_nowmaster_ai_summary)
    SN-->>Pub: summary sys_id/number
  end
  Worker->>Audit: append(...)

14.4 Governance Change Lifecycle (Admin)

sequenceDiagram
  autonumber
  participant Admin as Operator
  participant API as /api/admin/*
  participant Val as Validators
  participant Repo as JPA Repository
  participant Runtime as Runtime Services

  Admin->>API: validate payload
  API->>Val: validate(...)
  Val-->>API: ok/errors
  Admin->>API: upsert/deactivate
  API->>Repo: save/update active flag
  Repo-->>API: persisted row
  Note over Runtime: Next resolve/pull cycle uses new active rows

15) Quick PowerShell API examples (Windows-safe)

Validate Record Profile

$body = @{
  profileJson = '{"record_type":"incident","use_case":"en_incident_complete_summary","persona_role":"default","display_values":"both","fields":["number","short_description"],"reference_fields":[],"journal":[],"attachments":"optional"}'
} | ConvertTo-Json

Invoke-RestMethod -Method Post `
  -Uri "http://localhost:8080/api/admin/record-profiles/validate" `
  -ContentType "application/json" `
  -Body $body

Resolve Prompt Template

Invoke-RestMethod `
  -Uri "http://localhost:8080/api/admin/prompt-templates/resolve?recordType=u_nowmaster_ai_summary&intent=en_incident_complete_summary&variant=default"

List Rulesets

Invoke-RestMethod `
  -Uri "http://localhost:8080/api/admin/rulesets?includeInactive=false"