Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add variables field to the v1beta3 eventtype spec #8445

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
+344 −0
Open

feat: add variables field to the v1beta3 eventtype spec #8445

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6baa454
feat: add variable definitions to the v1beta3 eventtype spec
Ahmed-Labs Feb 4, 2025
7d00dad
fix: add omitempty to optional fields
Ahmed-Labs Feb 4, 2025
c6f6c9c
fix: update variables description and fix typo
Ahmed-Labs Feb 4, 2025
71b4b68
fix: formatting
Ahmed-Labs Feb 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions config/core/resources/eventtype.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,23 @@ spec:
value:
type: string
description: "Value of the attribute. May be a template string using curly brackets {} to represent variable sections of the string."
variables:
description: "Each variable used within attribute values must be defined in the variables field, except for unnamed variables (i.e. {}), which do not need to be defined."
type: array
items:
type: object
required:
- name
properties:
name:
type: string
description: "Name of the variable used within EventType attribute values enclosed in curly brackets."
pattern:
type: string
description: "A CESQL LIKE pattern that the attribute value would adhere to."
example:
type: string
description: "Example of an attribute value that adheres to the CESQL pattern."
status:
description: 'Status represents the current state of the EventType. This data
may be out of date.'
Expand Down
78 changes: 78 additions & 0 deletions docs/eventing-api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4191,6 +4191,20 @@ string
<p>Attributes is an array of CloudEvent attributes and extension attributes.</p>
</td>
</tr>
<tr>
<td>
<code>variables</code><br/>
<em>
<a href="#eventing.knative.dev/v1beta3.EventVariableDefinition">
[]EventVariableDefinition
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>Variables is an array that provides definitions for variables used within attribute values.</p>
</td>
</tr>
</table>
</td>
</tr>
Expand Down Expand Up @@ -4320,6 +4334,20 @@ string
<p>Attributes is an array of CloudEvent attributes and extension attributes.</p>
</td>
</tr>
<tr>
<td>
<code>variables</code><br/>
<em>
<a href="#eventing.knative.dev/v1beta3.EventVariableDefinition">
[]EventVariableDefinition
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>Variables is an array that provides definitions for variables used within attribute values.</p>
</td>
</tr>
</tbody>
</table>
<h3 id="eventing.knative.dev/v1beta3.EventTypeStatus">EventTypeStatus
Expand Down Expand Up @@ -4358,6 +4386,56 @@ knative.dev/pkg/apis/duck/v1.Status
</tr>
</tbody>
</table>
<h3 id="eventing.knative.dev/v1beta3.EventVariableDefinition">EventVariableDefinition
</h3>
<p>
(<em>Appears on:</em><a href="#eventing.knative.dev/v1beta3.EventTypeSpec">EventTypeSpec</a>)
</p>
<p>
</p>
<table>
<thead>
<tr>
<th>Field</th>
<th>Description</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<code>name</code><br/>
<em>
string
</em>
</td>
<td>
<p>Name is the name of the variable used within EventType attribute values enclosed in curly brackets.</p>
</td>
</tr>
<tr>
<td>
<code>pattern</code><br/>
<em>
string
</em>
</td>
<td>
<p>Pattern is a CESQL LIKE pattern that the attribute value would adhere to.</p>
</td>
</tr>
<tr>
<td>
<code>example</code><br/>
<em>
string
</em>
</td>
<td>
<p>Example is an example of an attribute value that adheres to the CESQL pattern.</p>
</td>
</tr>
</tbody>
</table>
<hr/>
<h2 id="flows.knative.dev/v1">flows.knative.dev/v1</h2>
<p>
Expand Down
12 changes: 12 additions & 0 deletions pkg/apis/eventing/v1beta3/eventtype_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,9 @@ type EventTypeSpec struct {
Description string `json:"description,omitempty"`
// Attributes is an array of CloudEvent attributes and extension attributes.
Attributes []EventAttributeDefinition `json:"attributes"`
// Variables is an array that provides definitions for variables used within attribute values.
// +optional
Variables []EventVariableDefinition `json:"variables,omitempty"`
Comment on lines +74 to +76
Copy link
Member

@pierDipi pierDipi Feb 25, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to ensure that every EventType version can be converted to and from v1beta3, so we need to add these new fields (and type) to every other served version (v1beta2 and v1beta1), otherwise the new fields will be dropped as the objects are stored using v1beta2:

eventing/config/core/resources/eventtype.yaml

Lines 171 to 173 in 71b4b68

- name: v1beta2
served: true
storage: true

Also we would need to update the validation functions for other versions too as it's done in the PR for v1beta3, so perhaps, we can extract common logic (like extract variables, etc) to a common file like ./pkg/apis/eventing/eventtype_validation_common.go

Once we've added the fields, types and validations to other versions, we need to update the conversion functions (for v1beta1 and v1beta2) like this one to set such new fields

https://github.com/knative/eventing/blob/71b4b68d3fd2e06d587fb05887027b3e6e15eb35/pkg/apis/eventing/v1beta1/eventtype_conversion.go

}

type EventAttributeDefinition struct {
Expand All @@ -86,6 +89,15 @@ type EventAttributeDefinition struct {
Value string `json:"value,omitempty"`
}

type EventVariableDefinition struct {
// Name is the name of the variable used within EventType attribute values enclosed in curly brackets.
Name string `json:"name"`
// Pattern is a CESQL LIKE pattern that the attribute value would adhere to.
Pattern string `json:"pattern,omitempty"`
// Example is an example of an attribute value that adheres to the CESQL pattern.
Example string `json:"example,omitempty"`
}

// EventTypeStatus represents the current state of a EventType.
type EventTypeStatus struct {
// inherits duck/v1 Status, which currently provides:
Expand Down
60 changes: 60 additions & 0 deletions pkg/apis/eventing/v1beta3/eventtype_validation.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ package v1beta3

import (
"context"
"strings"

"knative.dev/pkg/apis"
"knative.dev/pkg/kmp"
Expand All @@ -32,6 +33,7 @@ func (ets *EventTypeSpec) Validate(ctx context.Context) *apis.FieldError {
// TODO: validate attribute with name=source is a valid URI
// TODO: validate attribute with name=schema is a valid URI
errs = errs.Also(ets.ValidateAttributes().ViaField("attributes"))
errs = errs.Also(ets.ValidateVariables().ViaField("variables"))
return errs
}

Expand Down Expand Up @@ -83,3 +85,61 @@ func (ets *EventTypeSpec) ValidateAttributes() *apis.FieldError {

return nil
}

func (ets *EventTypeSpec) ValidateVariables() *apis.FieldError {
var errs *apis.FieldError

usedVariables := ets.extractAttributeVariables()
if len(usedVariables) == 0 {
return nil
}

definedVariables := make(map[string]EventVariableDefinition, len(ets.Variables))
for _, variable := range ets.Variables {
definedVariables[variable.Name] = variable
}

var missingVariables []string
for _, varName := range usedVariables {
if _, ok := definedVariables[varName]; !ok {
// keep track of any used variables that aren't defined
missingVariables = append(missingVariables, varName)
}
}

if len(missingVariables) > 0 {
errs = errs.Also(apis.ErrMissingField(missingVariables...))
}

return errs
}

// extractEmbeddedAttributeVariables extracts variables embedded within attribute values
// enclosed in curly brackets (e.g. "path.{A}.{B}" -> ["A", "B"]).
func (ets *EventTypeSpec) extractAttributeVariables() []string {
var variables []string

for _, attr := range ets.Attributes {
for idx := 0; idx < len(attr.Value); idx++ {
if attr.Value[idx] == '\\' {
idx++ // skip over escaped character
continue
}
if attr.Value[idx] != '{' {
continue // ignore characters not enclosed in curly brackets
}

idx++
var varName strings.Builder
for idx < len(attr.Value) && attr.Value[idx] != '}' {
varName.WriteByte(attr.Value[idx])
idx++
}

if idx < len(attr.Value) && attr.Value[idx] == '}' && varName.Len() > 0 {
variables = append(variables, varName.String())
}
}
}
return variables
}
156 changes: 156 additions & 0 deletions pkg/apis/eventing/v1beta3/eventtype_validation_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,109 @@ func TestEventTypeSpecValidation(t *testing.T) {
},
},
},
}, {
name: "invalid eventtype due to missing variable definitions",
ets: &EventTypeSpec{
Comment on lines +153 to +155
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this be a valid and interesting case to test as well? test.{test\\{Variable}.{defined\\}Variable}

Reference: &duckv1.KReference{
APIVersion: "eventing.knative.dev/v1",
Kind: "Broker",
Name: "test-broker",
},
Attributes: []EventAttributeDefinition{
{
Name: "type",
Value: "event-type",
Required: true,
},
{
Name: "source",
Value: testSource.String(),
Required: true,
},
{
Name: "specversion",
Value: "v1",
Required: true,
},
{
Name: "id",
Required: true,
},
{
Name: "attributeWithVariable",
Value: "test.{testVariable}.{definedVariable}",
Required: true,
},
{
Name: "anotherAttributeWithVariable",
Value: "test.something.{requestStatus}.more",
Required: true,
},
},
Variables: []EventVariableDefinition{
{
Name: "definedVariable",
Pattern: "%",
Example: "",
},
},
},
want: func() *apis.FieldError {
fe := apis.ErrMissingField("variables.requestStatus", "variables.testVariable")
return fe
}(),
}, {
name: "valid eventtype with variable definitions",
ets: &EventTypeSpec{
Reference: &duckv1.KReference{
APIVersion: "eventing.knative.dev/v1",
Kind: "Broker",
Name: "test-broker",
},
Attributes: []EventAttributeDefinition{
{
Name: "type",
Value: "event-type",
Required: true,
},
{
Name: "source",
Value: testSource.String(),
Required: true,
},
{
Name: "specversion",
Value: "v1",
Required: true,
},
{
Name: "id",
Required: true,
},
{
Name: "attributeWithVariable",
Value: "test.{testVariable}",
Required: true,
},
{
Name: "anotherAttributeWithVariable",
Value: "test.something.{requestStatus}.more",
Required: true,
},
},
Variables: []EventVariableDefinition{
{
Name: "testVariable",
Pattern: "%",
Example: "Any.value_passes",
},
{
Name: "requestStatus",
Pattern: "req_est.%",
Example: "request.failed",
},
},
},
},
}

Expand Down Expand Up @@ -491,3 +594,56 @@ func TestEventTypeImmutableFields(t *testing.T) {
})
}
}

func TestEventTypeSpecExtractAttributeVariables(t *testing.T) {
tests := []struct {
name string
ets *EventTypeSpec
want []string
}{
{
name: "extract included variables",
ets: &EventTypeSpec{
Attributes: []EventAttributeDefinition{
{
Name: "type",
Value: "a.{first}.{second}.{third}.{}",
Required: true,
},
{
Name: "source",
Value: "{fourth}{fifth}.ab.{sixth}",
Required: true,
},
},
},
want: []string{"first", "second", "third", "fourth", "fifth", "sixth"},
},
{
name: "ignores escaped curly brackets",
ets: &EventTypeSpec{
Attributes: []EventAttributeDefinition{
{
Name: "type",
Value: "a.{first}.\\{second}.{third}",
Required: true,
},
{
Name: "source",
Value: "\\{fourth}{fifth}.ab.{sixth}",
Required: true,
},
},
},
want: []string{"first", "third", "fifth", "sixth"},
},
}
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
got := test.ets.extractAttributeVariables()
if diff := cmp.Diff(got, test.want); diff != "" {
t.Error("ExtractAttributeVariables (-want, +got) =", diff)
}
})
}
}
Loading
Loading