Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Acquired interface query/subscription filters #20840

Draft
wants to merge 39 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
8dff044
Initial allowing view mismatches & new interfaces
dylant-da Feb 27, 2025
537ca00
Update test names for Upgrade checks (participant and compiler)
dylant-da Feb 27, 2025
f9a5ae0
Drop unused notANewTemplate predicate
dylant-da Feb 27, 2025
3570e83
lint
dylant-da Feb 27, 2025
11455e1
Test engine behaviour for acquired interfaces with package preference
dylant-da Mar 3, 2025
fc246e1
Add warning flag for added interface
dylant-da Mar 3, 2025
de9d6a0
Move testing for new flag to existing interface tests
dylant-da Mar 3, 2025
cd9a50f
Lint
dylant-da Mar 3, 2025
4bda213
Update security evidence
dylant-da Mar 3, 2025
227f71d
DO NOT MERGE - basic impl relaxing iface lookup
dylant-da Mar 4, 2025
b24a6a4
Drop view for source template since it might not exist DO NOT MERGE
dylant-da Mar 4, 2025
29057a6
Lookup interface view when it is available
dylant-da Mar 5, 2025
ba50621
Revert fetchTemplateG/fetchTemplateViaPkgId to prior fetchTemplate
dylant-da Mar 5, 2025
da2bcd4
lint
dylant-da Mar 5, 2025
d838fcb
Add tests for exercise on V1 contract picking up V2 interface instance
dylant-da Mar 5, 2025
8e77b92
Add failing test for queryNewInterfaceUpgrades
dylant-da Mar 5, 2025
e757f31
Add V3 to upgrade tests, and test that interface fetch works
dylant-da Mar 6, 2025
d36d770
Drop view check warning and unhelpful line in new implementation warning
dylant-da Mar 6, 2025
c6d9032
Drop interface view lookup / validation in fetchInterface entirely
dylant-da Mar 7, 2025
5ef9731
Inline hardFetchTemplate
dylant-da Mar 10, 2025
c6cb1b5
Fix fetchInterface to check destination contract, combine fetch impls
dylant-da Mar 10, 2025
65e8bef
lint
dylant-da Mar 10, 2025
ac0e4ed
Reword "added iface" msg, update "instance added" tests' expected msg
dylant-da Mar 10, 2025
dde28d4
Test that fetches/exercises by interface no longer calculate views
dylant-da Mar 11, 2025
6df1aeb
Drop prettyViewMismatch
dylant-da Mar 11, 2025
f8871dc
Assert that package preference affects view calculation
dylant-da Mar 11, 2025
93d2825
Remove view throw tests from ExceptionTest
dylant-da Mar 11, 2025
d688702
Test views with errors aren't evaluated in fetch/interface (SBuiltin)
dylant-da Mar 11, 2025
f4f05c1
Test views with errors aren't evaluated in fetch/interface (Evaluation)
dylant-da Mar 11, 2025
d4b8106
Update security evidence
dylant-da Mar 11, 2025
205b454
Drop accidentally committed changes to sandbox bootstrap script
dylant-da Mar 11, 2025
179bc57
Add back contract active/visibility checks that fetchInterface had
dylant-da Mar 11, 2025
d397b43
lint
dylant-da Mar 11, 2025
e98d45b
Update security evidence again
dylant-da Mar 11, 2025
8771d96
lint
dylant-da Mar 11, 2025
2fbb855
update docs
dylant-da Mar 11, 2025
f4ed42d
Implement acquired interface filters in the simplest way possible
dylant-da Mar 7, 2025
9fd38a8
Remove debugging logging statements
dylant-da Mar 7, 2025
6026b07
Mark queryInterfaceContractId test as working
dylant-da Mar 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -152,9 +152,6 @@ object RejectionGenerators {
) =>
CommandExecutionErrors.Interpreter.UpgradeError.DowngradeDropDefinedField
.Reject(renderedMessage, error)
case LfInterpretationError.Upgrade(error: LfInterpretationError.Upgrade.ViewMismatch) =>
CommandExecutionErrors.Interpreter.UpgradeError.ViewMismatch
.Reject(renderedMessage, error)
case LfInterpretationError.Upgrade(
error: LfInterpretationError.Upgrade.DowngradeFailed
) =>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -741,7 +741,7 @@ object IndexServiceImpl {
eventFormat: EventFormat,
alwaysPopulateArguments: Boolean,
)(implicit
contextualizedErrorLogger: ContextualizedErrorLogger
contextualizedErrorLogger: ContextualizedErrorLogger,
): () => Option[(TemplatePartiesFilter, EventProjectionProperties)] = {
@volatile var metadata: PackageMetadata = null
@volatile var filters: Option[(TemplatePartiesFilter, EventProjectionProperties)] = None
Expand Down Expand Up @@ -777,7 +777,7 @@ object IndexServiceImpl {
val eventProjectionProperties = EventProjectionProperties(
eventFormat = eventFormat,
interfaceImplementedBy =
interfaceId => metadata.interfacesImplementedBy.getOrElse(interfaceId, Set.empty),
interfaceId => interfacesImplementedByWithUpgrades(metadata, interfaceId),
resolveTypeConRef = metadata.resolveTypeConRef,
alwaysPopulateArguments = alwaysPopulateArguments,
)
Expand All @@ -793,14 +793,22 @@ object IndexServiceImpl {
}
}

private def interfacesImplementedByWithUpgrades(metadata: PackageMetadata, interfaceId: Ref.Identifier): Set[(Ref.Identifier, Ref.Identifier)] =
for {
originalImplementor <- metadata.interfacesImplementedBy.getOrElse(interfaceId, Set.empty)
(packageName, _) <- metadata.packageIdVersionMap.get(originalImplementor.packageId).toSet
implementor <- metadata.resolveTypeConRef(Ref.TypeConRef(Ref.PackageRef.Name(packageName), originalImplementor.qualifiedName))
} yield (originalImplementor, implementor)


private def templateIds(
metadata: PackageMetadata,
cumulativeFilter: CumulativeFilter,
): Set[Identifier] = {
val fromInterfacesDefs = cumulativeFilter.interfaceFilters.view
.map(_.interfaceTypeRef)
.flatMap(metadata.resolveTypeConRef(_))
.flatMap(metadata.interfacesImplementedBy.getOrElse(_, Set.empty).view)
.flatMap(interfacesImplementedByWithUpgrades(metadata, _).map(_._2).view)
.toSet

val fromTemplateDefs = cumulativeFilter.templateFilters.view
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
// Copyright (c) 2025 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
// Copyright (c) 2025 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved
// SPDX-License-Identifier: Apache-2.0

package com.digitalasset.canton.platform.store.dao
Expand Down Expand Up @@ -52,7 +52,7 @@ final case class EventProjectionProperties(
object EventProjectionProperties {

final case class Projection(
interfaces: Set[Identifier] = Set.empty,
interfaces: Set[(Identifier, Identifier)] = Set.empty,
createdEventBlob: Boolean = false,
contractArguments: Boolean = false,
) {
Expand All @@ -75,10 +75,10 @@ object EventProjectionProperties {
*/
def apply(
eventFormat: EventFormat,
interfaceImplementedBy: Identifier => Set[Identifier],
interfaceImplementedBy: Identifier => Set[(Identifier, Identifier)],
resolveTypeConRef: TypeConRef => Set[Identifier],
alwaysPopulateArguments: Boolean,
): EventProjectionProperties =
): EventProjectionProperties = {
EventProjectionProperties(
verbose = eventFormat.verbose,
templateWildcardWitnesses = templateWildcardWitnesses(eventFormat, alwaysPopulateArguments),
Expand All @@ -90,6 +90,7 @@ object EventProjectionProperties {
resolveTypeConRef,
),
)
}

private def templateWildcardWitnesses(
apiTransactionFilter: EventFormat,
Expand Down Expand Up @@ -141,7 +142,7 @@ object EventProjectionProperties {

private def witnessTemplateProjections(
apiEventFormat: EventFormat,
interfaceImplementedBy: Identifier => Set[Identifier],
interfaceImplementedBy: Identifier => Set[(Identifier, Identifier)],
resolveTypeConRef: TypeConRef => Set[Identifier],
): Map[Option[String], Map[Identifier, Projection]] = {
val partyFilterPairs =
Expand All @@ -155,9 +156,9 @@ object EventProjectionProperties {
val interfaceFilterProjections = for {
interfaceFilter <- cumulativeFilter.interfaceFilters.view
interfaceId <- resolveTypeConRef(interfaceFilter.interfaceTypeRef)
implementor <- interfaceImplementedBy(interfaceId).view
(originalImplementor, implementor) <- interfaceImplementedBy(interfaceId).view
} yield implementor -> Projection(
interfaces = if (interfaceFilter.includeView) Set(interfaceId) else Set.empty,
interfaces = if (interfaceFilter.includeView) Set((originalImplementor, interfaceId)) else Set.empty,
createdEventBlob = interfaceFilter.includeCreatedEventBlob,
contractArguments = false,
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -438,13 +438,13 @@ final class LfValueTranslation(
.map(toContractKeyApi(verbose))
)
def asyncInterfaceViews =
MonadUtil.sequentialTraverse(renderResult.interfaces.toList)(interfaceId =>
MonadUtil.sequentialTraverse(renderResult.interfaces.toList)({ case (originalImplementorTemplateId, interfaceId) =>
computeInterfaceView(
templateId,
originalImplementorTemplateId,
value.unversioned,
interfaceId,
).flatMap(toInterfaceView(eventProjectionProperties.verbose, interfaceId))
)
})

def asyncCreatedEventBlob = condFuture(renderResult.createdEventBlob) {
(for {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -802,34 +802,6 @@ object CommandExecutionErrors extends CommandExecutionErrorGroup {
}
}

@Explanation("View mismatch when trying to upgrade the contract")
@Resolution(
"Verify that the interface views of the contract have not changed"
)
object ViewMismatch
extends ErrorCode(
id = "INTERPRETATION_UPGRADE_ERROR_VIEW_MISMATCH",
ErrorCategory.InvalidGivenCurrentSystemStateOther,
) {
final case class Reject(
override val cause: String,
err: LfInterpretationError.Upgrade.ViewMismatch,
)(implicit
loggingContext: ContextualizedErrorLogger
) extends DamlErrorWithDefiniteAnswer(
cause = cause
) {

override def resources: Seq[(ErrorResource, String)] =
Seq(
(ErrorResource.ContractId, err.coid.coid),
(ErrorResource.InterfaceId, err.iterfaceId.toString),
(ErrorResource.TemplateId, err.srcTemplateId.toString),
(ErrorResource.TemplateId, err.dstTemplateId.toString),
)
}
}

@Explanation(
"An optional contract field with a value of Some may not be dropped during downgrading"
)
Expand Down
23 changes: 21 additions & 2 deletions sdk/compiler/daml-lf-tools/src/DA/Daml/LF/TypeChecker/Error.hs
Original file line number Diff line number Diff line change
Expand Up @@ -225,7 +225,6 @@ data UnwarnableError
| EUpgradeTemplateAddedKey !TypeConName !TemplateKey
| EUpgradeTriedToUpgradeIface !TypeConName
| EUpgradeMissingImplementation !TypeConName !TypeConName
| EForbiddenNewImplementation !TypeConName !TypeConName
| EUpgradeDependenciesFormACycle ![(PackageId, PackageMetadata)]
| EUpgradeMultiplePackagesWithSameNameAndVersion !PackageName !RawPackageVersion ![PackageId]
| EUpgradeTriedToUpgradeException !TypeConName
Expand Down Expand Up @@ -259,6 +258,7 @@ data ErrorOrWarning
| WETemplateInterfaceDependsOnScript !(PackageId, PackageMetadata)
-- ^ We want to recommend not uploading daml-script to ledger, but it's not a strict requirement
-- This we make it a warning on packages that define "engine entry points", i.e. templates or interfaces
| WEForbiddenNewImplementation !TypeConName !TypeConName
deriving (Eq, Show)

instance Pretty ErrorOrWarning where
Expand Down Expand Up @@ -317,6 +317,11 @@ instance Pretty ErrorOrWarning where
, "It is recommended that scripts/tests are defined in a separate package to your templates, and that"
, "you remove `" <> pprintDep dep <> "` from the dependencies in your daml.yaml"
]
WEForbiddenNewImplementation tpl iface ->
vcat
[ "Implementation of interface " <> pPrint iface <> " by template " <> pPrint tpl <> " is defined in this package, but not in the package that is being upgraded."
, "Only turn off this error if you know what you're doing."
]
where
withMismatchInfo :: [Mismatch UpgradeMismatchReason] -> Doc ann -> Doc ann
withMismatchInfo [] doc = doc
Expand Down Expand Up @@ -346,6 +351,7 @@ damlWarningFlagParserTypeChecker = DamlWarningFlagParser
, (unusedDependencyName, unusedDependencyFlag)
, (ownUpgradeDependencyName, ownUpgradeDependencyFlag)
, (templateInterfaceDependsOnScriptName, templateInterfaceDependsOnScriptFlag)
, (templateHasNewInterfaceInstanceName, templateHasNewInterfaceInstanceFlag)
]
, dwfpDefault = \case
WEUpgradeShouldDefineIfacesAndTemplatesSeparately {} -> AsError
Expand All @@ -366,6 +372,7 @@ damlWarningFlagParserTypeChecker = DamlWarningFlagParser
WEUnusedDependency {} -> AsWarning
WEOwnUpgradeDependency {} -> AsError
WETemplateInterfaceDependsOnScript {} -> AsWarning
WEForbiddenNewImplementation {} -> AsError
}

filterNameForErrorOrWarning :: ErrorOrWarning -> Maybe String
Expand All @@ -378,6 +385,7 @@ filterNameForErrorOrWarning err | couldNotExtractUpgradedExpressionFilter err =
filterNameForErrorOrWarning err | unusedDependencyFilter err = Just unusedDependencyName
filterNameForErrorOrWarning err | ownUpgradeDependencyFilter err = Just ownUpgradeDependencyName
filterNameForErrorOrWarning err | templateInterfaceDependsOnScriptFilter err = Just templateInterfaceDependsOnScriptName
filterNameForErrorOrWarning err | templateHasNewInterfaceInstanceFilter err = Just templateHasNewInterfaceInstanceName
filterNameForErrorOrWarning _ = Nothing

upgradedTemplateChangedFlag :: DamlWarningFlagStatus -> DamlWarningFlag ErrorOrWarning
Expand Down Expand Up @@ -497,6 +505,18 @@ templateInterfaceDependsOnScriptFilter =
WETemplateInterfaceDependsOnScript {} -> True
_ -> False

templateHasNewInterfaceInstanceFlag :: DamlWarningFlagStatus -> DamlWarningFlag ErrorOrWarning
templateHasNewInterfaceInstanceFlag status = RawDamlWarningFlag templateHasNewInterfaceInstanceName status templateHasNewInterfaceInstanceFilter

templateHasNewInterfaceInstanceName :: String
templateHasNewInterfaceInstanceName = "template-has-new-interface-instance"

templateHasNewInterfaceInstanceFilter :: ErrorOrWarning -> Bool
templateHasNewInterfaceInstanceFilter =
\case
WEForbiddenNewImplementation {} -> True
_ -> False

data PackageUpgradeOrigin = UpgradingPackage | UpgradedPackage
deriving (Eq, Ord, Show)

Expand Down Expand Up @@ -935,7 +955,6 @@ instance Pretty UnwarnableError where
EUpgradeTemplateAddedKey template _key -> "The upgraded template " <> pPrint template <> " cannot add a key where it didn't have one previously."
EUpgradeTriedToUpgradeIface iface -> "Tried to upgrade interface " <> pPrint iface <> ", but interfaces cannot be upgraded. They should be removed whenever a package is being upgraded."
EUpgradeMissingImplementation tpl iface -> "Implementation of interface " <> pPrint iface <> " by template " <> pPrint tpl <> " appears in package that is being upgraded, but does not appear in this package."
EForbiddenNewImplementation tpl iface -> "Implementation of interface " <> pPrint iface <> " by template " <> pPrint tpl <> " appears in this package, but does not appear in package that is being upgraded."
EUpgradeDependenciesFormACycle deps ->
vcat
[ "Dependencies from the `upgrades:` field and dependencies defined on the current package form a cycle:"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -546,13 +546,13 @@ checkAddedInstances ::
Module ->
HMS.HashMap (TypeConName, Qualified TypeConName) (Template, TemplateImplements) ->
TcUpgradeM ()
checkAddedInstances module_ instances = throwIfNonEmpty handleError instances
checkAddedInstances module_ instances = throwIfNonEmpty mkWarning instances
where
handleError ::
(Template, TemplateImplements) -> (Maybe Context, UnwarnableError)
handleError (tpl, impl) =
mkWarning ::
(Template, TemplateImplements) -> (Maybe Context, ErrorOrWarning)
mkWarning (tpl, impl) =
( Just (ContextTemplate module_ tpl TPWhole)
, EForbiddenNewImplementation (NM.name tpl) (LF.qualObject (NM.name impl))
, WEForbiddenNewImplementation (NM.name tpl) (LF.qualObject (NM.name impl))
)

-- It is always invalid to keep an interface in an upgrade
Expand Down
7 changes: 3 additions & 4 deletions sdk/compiler/damlc/tests/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -455,10 +455,9 @@ da_haskell_test(
"//test-common:upgrades-FailsWhenATopLevelVariantAddsAFieldToAConstructorsType-v2.dar",
"//test-common:upgrades-FailsWhenATopLevelVariantRemovesAConstructor-v1.dar",
"//test-common:upgrades-FailsWhenATopLevelVariantRemovesAConstructor-v2.dar",
"//test-common:upgrades-FailsWhenAnInstanceIsAddedSeparateDep-v1.dar",
"//test-common:upgrades-FailsWhenAnInstanceIsAddedSeparateDep-v2.dar",
"//test-common:upgrades-FailsWhenAnInstanceIsAddedUpgradedPackage-v1.dar",
"//test-common:upgrades-FailsWhenAnInstanceIsAddedUpgradedPackage-v2.dar",
"//test-common:upgrades-SucceedsWhenAnInstanceIsAddedSeparateDep-files",
"//test-common:upgrades-SucceedsWhenAnInstanceIsAddedSeparateDep-dep.dar",
"//test-common:upgrades-SucceedsWhenAnInstanceIsAddedUpgradedPackage-files",
"//test-common:upgrades-FailsWhenAnInstanceIsDropped-v1.dar",
"//test-common:upgrades-FailsWhenAnInstanceIsDropped-v2.dar",
"//test-common:upgrades-FailsWhenAnInterfaceIsDefinedInAnUpgradingPackageWhenItWasAlreadyInThePriorPackage-v1.dar",
Expand Down
Loading