Document Validation¶
Validations¶
The validation system provides a unified approach to complex validations that require coordination of multiple documents and business logic that resides in consumer services.
Deckhand focuses on two types of validations: schema validations and policy validations.
Deckhand-Provided Validations¶
Deckhand provides a few internal validations which are made available
immediately upon document ingestion. Deckhand’s internal schema validations are
defined as DataSchema
documents.
Here is a list of internal validations:
deckhand-document-schema-validation
- All concrete documents in the revision successfully pass their JSON schema validations. Will cause this to report an error.
Externally Provided Validations¶
As mentioned, other services can report whether named validations that have
been registered by those services as success or failure. DataSchema
control
documents are used to register a new validation mapping that other services
can reference to verify whether a Deckhand bucket is in a valid configuration.
For more information, refer to the DataSchema
section in
Document Types.
Validation Codes¶
- D001 - Indicates document sanity-check validation failure pre- or post-rendering. This means that the document structure is fundamentally broken.
- D002 - Indicates document post-rendering validation failure. This means
that after a document has rendered, the document may fail validation.
For example, if a
DataSchema
document for a given revision indicates that.data.a
is a required field but a layering action during rendering deletes.data.a
, then post-rendering validation will necessarily fail. This implies a conflict in the set of document requirements.
Schema Validations¶
Schema validations are controlled by two mechanisms:
- Deckhand’s internal schema validation for sanity-checking the formatting
of the default documents that it understands. For example, Deckhand
will check that a
LayeringPolicy
,ValidationPolicy
orDataSchema
adhere to the appropriate internal schemas. - Externally provided validations via
DataSchema
documents. These documents can be registered by external services and subject the target document’sdata
section to additional validations, validations specified by thedata
section of theDataSchema
document.
Validation Stages¶
Deckhand performs pre- and post-rendering validation on documents.
Pre-Rendering¶
Carried out during document ingestion.
For pre-rendering validation 3 types of behavior are possible:
- Successfully validated documents are stored in Deckhand’s database.
- Failure to validate a document’s basic properties will result in a 400 Bad Request error getting raised.
- Failure to validate a document’s schema-specific properties will result in a validation error created in the database, which can be later returned via the Validations API.
Post-Rendering¶
Carried out after rendering all documents.
For post-rendering validation, 2 types of behavior are possible:
- Successfully validated post-rendered documents are returned to the user.
- Failure to validate post-rendered documents results in a 500 Internal Server Error getting raised.
Validation Schemas¶
Below are the schemas Deckhand uses to validate documents.
Base Schema¶
Base schema.
Base JSON schema against which all Deckhand documents are validated.
--- schema: deckhand/DataSchema/v1 metadata: name: deckhand/Base/v1 schema: metadata/Control/v1 data: $schema: http://json-schema.org/schema# properties: schema: type: string pattern: ^[A-Za-z]+/[A-Za-z]+/v\d+$ metadata: # True validation of the metadata section will be done using # the schema specfied in the metadata section type: object properties: name: type: string schema: anyOf: - type: string pattern: ^metadata/Document/v\d+$ - type: string pattern: ^metadata/Control/v\d+$ additionalProperties: true required: - 'name' - 'schema' # This schema should allow anything in the data section data: type: - 'null' - 'string' - 'object' - 'array' - 'number' - 'boolean' additionalProperties: false required: - schema - metadata - data
This schema is used to sanity-check all documents that are passed to Deckhand. Failure to pass this schema results in a critical error.
Metadata Schemas¶
Metadata schemas validate the metadata
section of every document
ingested by Deckhand.
Metadata Control
schema.JSON schema against which the metadata section of each
metadata/Control
document type is validated. Applies to all static documents meant to configure Deckhand behavior, like LayeringPolicy, ValidationPolicy, and DataSchema documents.labels: type: object additionalProperties: type: string additionalProperties: true required: - schema - name # NOTE(felipemonteiro): layeringDefinition is not needed for any control # documents as neither LayeringPolicy, ValidationPolicy or DataSchema # documents are ever layered together.
Metadata Document
schema.JSON schema against which the metadata section of each
metadata/Document
document type is validated. Applies to all site definition documents or “regular” documents that require rendering.- actions actions_requires_parent_selector: dependencies: # Requires that if actions are provided, then so too must # parentSelector. actions: required: - parentSelector substitution_dest: type: object properties: path: type: string pattern: type: string recurse: type: object properties: depth: type: integer minimum: -1 # -1 indicates that the recursion depth is infinite. Refinements # to this value should be specified by the caller. default: -1 required: - depth additionalProperties: false required: - path type: object properties: schema: type: string pattern: ^metadata/Document/v\d+$ name: type: string labels: type: object replacement: type: boolean layeringDefinition: type: object properties: layer: type: string abstract: type: boolean parentSelector: type: object minProperties: 1 actions: type: array minItems: 1 items: type: object properties: method: enum: - replace - delete - merge path: type: string additionalProperties: false required: - method - path additionalProperties: false required: - 'layer' allOf: - $ref: "#/definitions/parent_selector_requires_actions" - $ref: "#/definitions/actions_requires_parent_selector" substitutions: type: array items: type: object properties: dest: anyOf: - $ref: "#/definitions/substitution_dest" - type: array minItems: 1 items: $ref: "#/definitions/substitution_dest" src: type: object properties: schema: type: string pattern: ^[A-Za-z]+/[A-Za-z]+/v\d+$ name: type: string path: type: string pattern: type: string match_group: type: integer additionalProperties: false required: - schema - name - path additionalProperties: false required: - dest - src storagePolicy: type: string enum: - encrypted - cleartext additionalProperties: false required: - schema - name - storagePolicy - layeringDefinition
Validation Schemas¶
DataSchema schemas validate the data
section of every document ingested
by Deckhand.
All schemas below are DataSchema
documents. They define additional
properties not included in the base schema or override default properties in
the base schema.
These schemas are only enforced after validation for the base schema has
passed. Failure to pass these schemas will result in an error entry being
created for the validation with name deckhand-schema-validation
corresponding to the created revision.
CertificateAuthorityKey
schema.JSON schema against which all documents with
deckhand/CertificateAuthorityKey/v1
schema are validated.--- schema: deckhand/DataSchema/v1 metadata: name: deckhand/CertificateAuthorityKey/v1 schema: metadata/Control/v1 data: $schema: http://json-schema.org/schema# type: string
This schema is used to sanity-check all CertificateAuthorityKey documents that are passed to Deckhand.
CertificateAuthority
schema.JSON schema against which all documents with
deckhand/CertificateAuthority/v1
schema are validated.--- schema: deckhand/DataSchema/v1 metadata: name: deckhand/CertificateAuthority/v1 schema: metadata/Control/v1 data: $schema: http://json-schema.org/schema# type: string
This schema is used to sanity-check all
CertificateAuthority
documents that are passed to Deckhand.CertificateKey
schema.JSON schema against which all documents with
deckhand/CertificateKey/v1
schema are validated.--- schema: deckhand/DataSchema/v1 metadata: name: deckhand/CertificateKey/v1 schema: metadata/Control/v1 data: $schema: http://json-schema.org/schema# type: string
This schema is used to sanity-check all
CertificateKey
documents that are passed to Deckhand.Certificate
schema.JSON schema against which all documents with
deckhand/Certificate/v1
schema are validated.--- schema: deckhand/DataSchema/v1 metadata: name: deckhand/Certificate/v1 schema: metadata/Control/v1 data: $schema: http://json-schema.org/schema# type: string
This schema is used to sanity-check all
Certificate
documents that are passed to Deckhand.LayeringPolicy
schema.JSON schema against which all documents with
deckhand/LayeringPolicy/v1
schema are validated.--- schema: deckhand/DataSchema/v1 metadata: name: deckhand/LayeringPolicy/v1 schema: metadata/Control/v1 data: $schema: http://json-schema.org/schema# type: object properties: layerOrder: type: array items: type: string additionalProperties: false required: - layerOrder
This schema is used to sanity-check all
LayeringPolicy
documents that are passed to Deckhand.PrivateKey
schema.JSON schema against which all documents with
deckhand/PrivateKey/v1
schema are validated.--- schema: deckhand/DataSchema/v1 metadata: name: deckhand/Passphrase/v1 schema: metadata/Control/v1 data: $schema: http://json-schema.org/schema# type: string
This schema is used to sanity-check all
PrivateKey
documents that are passed to Deckhand.PublicKey
schema.JSON schema against which all documents with
deckhand/PublicKey/v1
schema are validated.--- schema: deckhand/DataSchema/v1 metadata: name: deckhand/PublicKey/v1 schema: metadata/Control/v1 data: $schema: http://json-schema.org/schema# type: string
This schema is used to sanity-check all
PublicKey
documents that are passed to Deckhand.Passphrase
schema.JSON schema against which all documents with
deckhand/Passphrase/v1
schema are validated.--- schema: deckhand/DataSchema/v1 metadata: name: deckhand/PrivateKey/v1 schema: metadata/Control/v1 data: $schema: http://json-schema.org/schema# type: string
This schema is used to sanity-check all
Passphrase
documents that are passed to Deckhand.ValidationPolicy
schema.JSON schema against which all documents with
deckhand/ValidationPolicy/v1
schema are validated.--- schema: deckhand/DataSchema/v1 metadata: name: deckhand/ValidationPolicy/v1 schema: metadata/Control/v1 data: $schema: http://json-schema.org/schema# type: object properties: validations: type: array items: type: object properties: name: type: string pattern: ^.*-(validation|verification)$ expiresAfter: type: string additionalProperties: false required: - name required: - validations additionalProperties: false
This schema is used to sanity-check all
ValidationPolicy
documents that are passed to Deckhand.