Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Duration is not correctly deserialized with Property v2 default value #5615

Closed
mgabelle opened this issue Oct 23, 2024 · 0 comments · Fixed by #5630
Closed

Duration is not correctly deserialized with Property v2 default value #5615

mgabelle opened this issue Oct 23, 2024 · 0 comments · Fixed by #5630
Assignees
@loicmathieu
Labels
area/backend Needs backend code changes bug Something isn't working

Comments

@mgabelle
Copy link
Contributor

Describe the issue

When testing the plugin Azure Datafactory locally I am able to run it when the duration property are set in the flow.
But I am getting a de-serialization issue when using the default properties.

Here's the PR with the code causing issue : PR

Here's some videos example of the issue :

Here's the stack trace :

java.lang.IllegalArgumentException: Cannot deserialize value of type `java.time.Duration` from String "5.000000000": Failed to deserialize java.time.Duration: (java.time.format.DateTimeParseException) Text cannot be parsed to a Duration
2024-10-23T08:19:54.961268349Z  at [Source: UNKNOWN; byte offset: #UNKNOWN]
2024-10-23T08:19:54.961274856Z 	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4636)
2024-10-23T08:19:54.961281495Z 	at com.fasterxml.jackson.databind.ObjectMapper.convertValue(ObjectMapper.java:4567)
2024-10-23T08:19:54.961288035Z 	at io.kestra.core.models.property.Property.as(Property.java:82)
2024-10-23T08:19:54.961294368Z 	at io.kestra.core.runners.DefaultRunContext.render(DefaultRunContext.java:210)
2024-10-23T08:19:54.961301152Z 	at io.kestra.plugin.azure.datafactory.CreateRun.run(CreateRun.java:167)
2024-10-23T08:19:54.961307687Z 	at io.kestra.plugin.azure.datafactory.CreateRun.run(CreateRun.java:52)
2024-10-23T08:19:54.961314143Z 	at io.kestra.core.runners.WorkerTaskCallable.doCall(WorkerTaskCallable.java:78)
2024-10-23T08:19:54.961320565Z 	at io.kestra.core.runners.AbstractWorkerCallable.call(AbstractWorkerCallable.java:56)
2024-10-23T08:19:54.961327187Z 	at io.kestra.core.runners.WorkerSecurityService.callInSecurityContext(WorkerSecurityService.java:10)
2024-10-23T08:19:54.961333770Z 	at io.kestra.core.runners.Worker.callJob(Worker.java:773)
2024-10-23T08:19:54.961357858Z 	at io.kestra.core.runners.Worker.runAttempt(Worker.java:728)
2024-10-23T08:19:54.961365543Z 	at io.kestra.core.runners.Worker.run(Worker.java:580)
2024-10-23T08:19:54.961396656Z 	at io.kestra.core.runners.Worker.handleTask(Worker.java:284)
2024-10-23T08:19:54.961403247Z 	at io.kestra.core.runners.Worker.lambda$run$7(Worker.java:240)
2024-10-23T08:19:54.961408074Z 	at io.micrometer.core.instrument.internal.TimedRunnable.run(TimedRunnable.java:49)
2024-10-23T08:19:54.961412587Z 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
2024-10-23T08:19:54.961417118Z 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
2024-10-23T08:19:54.961421681Z 	at java.base/java.lang.Thread.run(Unknown Source)
2024-10-23T08:19:54.961428718Z Caused by: com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `java.time.Duration` from String "5.000000000": Failed to deserialize java.time.Duration: (java.time.format.DateTimeParseException) Text cannot be parsed to a Duration
2024-10-23T08:19:54.961434833Z  at [Source: UNKNOWN; byte offset: #UNKNOWN]
2024-10-23T08:19:54.961439264Z 	at com.fasterxml.jackson.databind.exc.InvalidFormatException.from(InvalidFormatException.java:67)
2024-10-23T08:19:54.961443735Z 	at com.fasterxml.jackson.databind.DeserializationContext.weirdStringException(DeserializationContext.java:1959)
2024-10-23T08:19:54.961448484Z 	at com.fasterxml.jackson.databind.DeserializationContext.handleWeirdStringValue(DeserializationContext.java:1245)
2024-10-23T08:19:54.961453060Z 	at com.fasterxml.jackson.datatype.jsr310.deser.JSR310DeserializerBase._handleDateTimeException(JSR310DeserializerBase.java:176)
2024-10-23T08:19:54.961457858Z 	at com.fasterxml.jackson.datatype.jsr310.deser.DurationDeserializer._fromString(DurationDeserializer.java:200)
2024-10-23T08:19:54.961462378Z 	at com.fasterxml.jackson.datatype.jsr310.deser.DurationDeserializer.deserialize(DurationDeserializer.java:163)
2024-10-23T08:19:54.961467011Z 	at com.fasterxml.jackson.datatype.jsr310.deser.DurationDeserializer.deserialize(DurationDeserializer.java:44)
2024-10-23T08:19:54.961471506Z 	at com.fasterxml.jackson.databind.ObjectMapper._convert(ObjectMapper.java:4631)
2024-10-23T08:19:54.961476109Z 	... 17 common frames omitted
2024-10-23T08:19:54.961480392Z Caused by: java.time.format.DateTimeParseException: Text cannot be parsed to a Duration
2024-10-23T08:19:54.961484900Z 	at java.base/java.time.Duration.parse(Unknown Source)
2024-10-23T08:19:54.961489191Z 	at com.fasterxml.jackson.datatype.jsr310.deser.DurationDeserializer._fromString(DurationDeserializer.java:198)
2024-10-23T08:19:54.961493734Z 	... 20 common frames omitted

Environment

  • Kestra Version: last version of kestra:develop
  • Operating System (OS/Docker/Kubernetes):
  • Java Version (if you don't run kestra in Docker):
@mgabelle mgabelle added the bug Something isn't working label Oct 23, 2024
@github-project-automation github-project-automation bot moved this to Backlog in Issues Oct 23, 2024
@mgabelle mgabelle added the area/backend Needs backend code changes label Oct 23, 2024
@loicmathieu loicmathieu self-assigned this Oct 23, 2024
loicmathieu added a commit that referenced this issue Oct 23, 2024
@loicmathieu
fix(core): serialize duration as strings
dfadb06 
Fixes #5615

Since Jackson 2.10, to serialize duration as string we need to switch WRITE_DURATIONS_AS_TIMESTAMPS off as it no longuer use WRITE_DATES_AS_TIMESTAMPS.
I tested and an old flow with a duration as timestamp is still readable so this is a backward compatible change.
@loicmathieu loicmathieu mentioned this issue Oct 23, 2024
Merged
loicmathieu added a commit that referenced this issue Oct 25, 2024
@loicmathieu
fix(core): serialize duration as strings
75d9110 
Fixes #5615

Since Jackson 2.10, to serialize duration as string we need to switch WRITE_DURATIONS_AS_TIMESTAMPS off as it no longuer use WRITE_DATES_AS_TIMESTAMPS.
I tested and an old flow with a duration as timestamp is still readable so this is a backward compatible change.
loicmathieu added a commit that referenced this issue Oct 25, 2024
@loicmathieu
fix(core): serialize duration as strings
a6c831d 
Fixes #5615

Since Jackson 2.10, to serialize duration as string we need to switch WRITE_DURATIONS_AS_TIMESTAMPS off as it no longuer use WRITE_DATES_AS_TIMESTAMPS.
I tested and an old flow with a duration as timestamp is still readable so this is a backward compatible change.
loicmathieu added a commit that referenced this issue Oct 28, 2024
@loicmathieu
fix(core): serialize duration as strings
63d70ac 
Fixes #5615

Since Jackson 2.10, to serialize duration as string we need to switch WRITE_DURATIONS_AS_TIMESTAMPS off as it no longuer use WRITE_DATES_AS_TIMESTAMPS.
I tested and an old flow with a duration as timestamp is still readable so this is a backward compatible change.

But for the State duration, we still need to serialize it as a number as the database expect a number
@github-project-automation github-project-automation bot moved this from Backlog to Done in Issues Oct 28, 2024
MilosPaunovic pushed a commit that referenced this issue Nov 4, 2024
@loicmathieu @MilosPaunovic
fix(core): serialize duration as strings
5132de7 
Fixes #5615

Since Jackson 2.10, to serialize duration as string we need to switch WRITE_DURATIONS_AS_TIMESTAMPS off as it no longuer use WRITE_DATES_AS_TIMESTAMPS.
I tested and an old flow with a duration as timestamp is still readable so this is a backward compatible change.

But for the State duration, we still need to serialize it as a number as the database expect a number
@tchiotludo tchiotludo mentioned this issue Nov 20, 2024
Closed
loicmathieu added a commit that referenced this issue Nov 25, 2024
@loicmathieu
fix(core): serialize duration as strings in Property
5263872 
Fixes #5615
loicmathieu added a commit that referenced this issue Nov 25, 2024
@loicmathieu
fix(core): serialize duration as strings in Property
9f4dbd8 
Fixes #5615
loicmathieu added a commit that referenced this issue Nov 25, 2024
@loicmathieu
fix(core): serialize duration as strings in Property
bc27733 
Fixes #5615
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@loicmathieu loicmathieu

Labels
area/backend Needs backend code changes bug Something isn't working
Projects
Issues
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(core): serialize duration as strings kestra-io/kestra
2 participants
@loicmathieu @mgabelle