Document Validation

Validations

The validation system provides a unified approach to complex validations that require coordination of multiple documents and business logic that resides in consumer services.

Deckhand focuses on two types of validations: schema validations and policy validations.

Deckhand-Provided Validations

Deckhand provides a few internal validations which are made available immediately upon document ingestion. Deckhand’s internal schema validations are defined as DataSchema documents.

Here is a list of internal validations:

  • deckhand-document-schema-validation - All concrete documents in the revision successfully pass their JSON schema validations. Will cause this to report an error.

Externally Provided Validations

As mentioned, other services can report whether named validations that have been registered by those services as success or failure. DataSchema control documents are used to register a new validation mapping that other services can reference to verify whether a Deckhand bucket is in a valid configuration. For more information, refer to the DataSchema section in Document Types.

Validation Codes

  • D001 - Indicates document sanity-check validation failure pre- or post-rendering. This means that the document structure is fundamentally broken.
  • D002 - Indicates document post-rendering validation failure. This means that after a document has rendered, the document may fail validation. For example, if a DataSchema document for a given revision indicates that .data.a is a required field but a layering action during rendering deletes .data.a, then post-rendering validation will necessarily fail. This implies a conflict in the set of document requirements.

Schema Validations

Schema validations are controlled by two mechanisms:

  1. Deckhand’s internal schema validation for sanity-checking the formatting of the default documents that it understands. For example, Deckhand will check that a LayeringPolicy, ValidationPolicy or DataSchema adhere to the appropriate internal schemas.
  2. Externally provided validations via DataSchema documents. These documents can be registered by external services and subject the target document’s data section to additional validations, validations specified by the data section of the DataSchema document.

Validation Stages

Deckhand performs pre- and post-rendering validation on documents.

Pre-Rendering

Carried out during document ingestion.

For pre-rendering validation 3 types of behavior are possible:

  1. Successfully validated documents are stored in Deckhand’s database.
  2. Failure to validate a document’s basic properties will result in a 400 Bad Request error getting raised.
  3. Failure to validate a document’s schema-specific properties will result in a validation error created in the database, which can be later returned via the Validations API.

Post-Rendering

Carried out after rendering all documents.

For post-rendering validation, 2 types of behavior are possible:

  1. Successfully validated post-rendered documents are returned to the user.
  2. Failure to validate post-rendered documents results in a 500 Internal Server Error getting raised.

Validation Schemas

Below are the schemas Deckhand uses to validate documents.

Base Schema

  • Base schema.

    Base JSON schema against which all Deckhand documents are validated.

    Base schema that applies to all documents.
    ---
    schema: deckhand/DataSchema/v1
    metadata:
      name: deckhand/Base/v1
      schema: metadata/Control/v1
    data:
      $schema: http://json-schema.org/schema#
      properties:
        schema:
          type: string
          pattern: ^[A-Za-z]+/[A-Za-z]+/v\d+$
        metadata:
          # True validation of the metadata section will be done using
          # the schema specfied in the metadata section
          type: object
          properties:
            name:
              type: string
            schema:
              anyOf:
                - type: string
                  pattern: ^metadata/Document/v\d+$
                - type: string
                  pattern: ^metadata/Control/v\d+$
          additionalProperties: true
          required:
            - 'name'
            - 'schema'
        # This schema should allow anything in the data section
        data:
          type:
            - 'null'
            - 'string'
            - 'object'
            - 'array'
            - 'number'
            - 'boolean'
      additionalProperties: false
      required:
        - schema
        - metadata
        - data
    

    This schema is used to sanity-check all documents that are passed to Deckhand. Failure to pass this schema results in a critical error.

Metadata Schemas

Metadata schemas validate the metadata section of every document ingested by Deckhand.

  • Metadata Control schema.

    JSON schema against which the metadata section of each metadata/Control document type is validated. Applies to all static documents meant to configure Deckhand behavior, like LayeringPolicy, ValidationPolicy, and DataSchema documents.

    Schema for metadata/Control metadata document sections.
        labels:
          type: object
          additionalProperties:
            type: string
      additionalProperties: true
      required:
        - schema
        - name
        # NOTE(felipemonteiro): layeringDefinition is not needed for any control
        # documents as neither LayeringPolicy, ValidationPolicy or DataSchema
        # documents are ever layered together.
    
  • Metadata Document schema.

    JSON schema against which the metadata section of each metadata/Document document type is validated. Applies to all site definition documents or “regular” documents that require rendering.

    Schema for metadata/Document metadata document sections.
                - actions
        actions_requires_parent_selector:
          dependencies:
            # Requires that if actions are provided, then so too must
            # parentSelector.
            actions:
              required:
                - parentSelector
        substitution_dest:
          type: object
          properties:
            path:
              type: string
            pattern:
              type: string
            recurse:
              type: object
              properties:
                depth:
                  type: integer
                  minimum: -1
                  # -1 indicates that the recursion depth is infinite. Refinements
                  # to this value should be specified by the caller.
                  default: -1
              required:
                - depth
          additionalProperties: false
          required:
            - path
      type: object
      properties:
        schema:
          type: string
          pattern: ^metadata/Document/v\d+$
        name:
          type: string
        labels:
          type: object
        replacement:
          type: boolean
        layeringDefinition:
          type: object
          properties:
            layer:
              type: string
            abstract:
              type: boolean
            parentSelector:
              type: object
              minProperties: 1
            actions:
              type: array
              minItems: 1
              items:
                type: object
                properties:
                  method:
                    enum:
                      - replace
                      - delete
                      - merge
                  path:
                    type: string
                additionalProperties: false
                required:
                  - method
                  - path
          additionalProperties: false
          required:
            - 'layer'
          allOf:
            - $ref: "#/definitions/parent_selector_requires_actions"
            - $ref: "#/definitions/actions_requires_parent_selector"
        substitutions:
          type: array
          items:
            type: object
            properties:
              dest:
                anyOf:
                  - $ref: "#/definitions/substitution_dest"
                  - type: array
                    minItems: 1
                    items:
                      $ref: "#/definitions/substitution_dest"
              src:
                type: object
                properties:
                  schema:
                    type: string
                    pattern: ^[A-Za-z]+/[A-Za-z]+/v\d+$
                  name:
                    type: string
                  path:
                    type: string
                  pattern:
                    type: string
                  match_group:
                    type: integer
                additionalProperties: false
                required:
                  - schema
                  - name
                  - path
            additionalProperties: false
            required:
              - dest
              - src
        storagePolicy:
          type: string
          enum:
            - encrypted
            - cleartext
      additionalProperties: false
      required:
        - schema
        - name
        - storagePolicy
        - layeringDefinition
    

Validation Schemas

DataSchema schemas validate the data section of every document ingested by Deckhand.

All schemas below are DataSchema documents. They define additional properties not included in the base schema or override default properties in the base schema.

These schemas are only enforced after validation for the base schema has passed. Failure to pass these schemas will result in an error entry being created for the validation with name deckhand-schema-validation corresponding to the created revision.

  • CertificateAuthorityKey schema.

    JSON schema against which all documents with deckhand/CertificateAuthorityKey/v1 schema are validated.

    Schema for CertificateAuthorityKey documents.
    ---
    schema: deckhand/DataSchema/v1
    metadata:
      name: deckhand/CertificateAuthorityKey/v1
      schema: metadata/Control/v1
    data:
      $schema: http://json-schema.org/schema#
      type: string
    

    This schema is used to sanity-check all CertificateAuthorityKey documents that are passed to Deckhand.

  • CertificateAuthority schema.

    JSON schema against which all documents with deckhand/CertificateAuthority/v1 schema are validated.

    Schema for CertificateAuthority documents.
    ---
    schema: deckhand/DataSchema/v1
    metadata:
      name: deckhand/CertificateAuthority/v1
      schema: metadata/Control/v1
    data:
      $schema: http://json-schema.org/schema#
      type: string
    

    This schema is used to sanity-check all CertificateAuthority documents that are passed to Deckhand.

  • CertificateKey schema.

    JSON schema against which all documents with deckhand/CertificateKey/v1 schema are validated.

    Schema for CertificateKey documents.
    ---
    schema: deckhand/DataSchema/v1
    metadata:
      name: deckhand/CertificateKey/v1
      schema: metadata/Control/v1
    data:
      $schema: http://json-schema.org/schema#
      type: string
    

    This schema is used to sanity-check all CertificateKey documents that are passed to Deckhand.

  • Certificate schema.

    JSON schema against which all documents with deckhand/Certificate/v1 schema are validated.

    Schema for Certificate documents.
    ---
    schema: deckhand/DataSchema/v1
    metadata:
      name: deckhand/Certificate/v1
      schema: metadata/Control/v1
    data:
      $schema: http://json-schema.org/schema#
      type: string
    

    This schema is used to sanity-check all Certificate documents that are passed to Deckhand.

  • LayeringPolicy schema.

    JSON schema against which all documents with deckhand/LayeringPolicy/v1 schema are validated.

    Schema for LayeringPolicy documents.
    ---
    schema: deckhand/DataSchema/v1
    metadata:
      name: deckhand/LayeringPolicy/v1
      schema: metadata/Control/v1
    data:
      $schema: http://json-schema.org/schema#
      type: object
      properties:
        layerOrder:
          type: array
          items:
            type: string
      additionalProperties: false
      required:
        - layerOrder
    

    This schema is used to sanity-check all LayeringPolicy documents that are passed to Deckhand.

  • PrivateKey schema.

    JSON schema against which all documents with deckhand/PrivateKey/v1 schema are validated.

    Schema for PrivateKey documents.
    ---
    schema: deckhand/DataSchema/v1
    metadata:
      name: deckhand/Passphrase/v1
      schema: metadata/Control/v1
    data:
      $schema: http://json-schema.org/schema#
      type: string
    

    This schema is used to sanity-check all PrivateKey documents that are passed to Deckhand.

  • PublicKey schema.

    JSON schema against which all documents with deckhand/PublicKey/v1 schema are validated.

    Schema for PublicKey documents.
    ---
    schema: deckhand/DataSchema/v1
    metadata:
      name: deckhand/PublicKey/v1
      schema: metadata/Control/v1
    data:
      $schema: http://json-schema.org/schema#
      type: string
    

    This schema is used to sanity-check all PublicKey documents that are passed to Deckhand.

  • Passphrase schema.

    JSON schema against which all documents with deckhand/Passphrase/v1 schema are validated.

    Schema for Passphrase documents.
    ---
    schema: deckhand/DataSchema/v1
    metadata:
      name: deckhand/PrivateKey/v1
      schema: metadata/Control/v1
    data:
      $schema: http://json-schema.org/schema#
      type: string
    

    This schema is used to sanity-check all Passphrase documents that are passed to Deckhand.

  • ValidationPolicy schema.

    JSON schema against which all documents with deckhand/ValidationPolicy/v1 schema are validated.

    Schema for ValidationPolicy documents.
    ---
    schema: deckhand/DataSchema/v1
    metadata:
      name: deckhand/ValidationPolicy/v1
      schema: metadata/Control/v1
    data:
      $schema: http://json-schema.org/schema#
      type: object
      properties:
        validations:
          type: array
          items:
            type: object
            properties:
              name:
                type: string
                pattern: ^.*-(validation|verification)$
              expiresAfter:
                type: string
            additionalProperties: false
            required:
              - name
      required:
        - validations
      additionalProperties: false
    

    This schema is used to sanity-check all ValidationPolicy documents that are passed to Deckhand.