Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: expose missedHeartbeatsLimit config through Spring Boot properties (addresses #633) #634

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 7 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -299,7 +299,7 @@ The number of executions fetched each time is equal to `(upperLimitFractionOfThr
Fetched executions are already locked/picked for this scheduler-instance thus saving one `UPDATE` statement.
<br/>For normal usage, set to for example `0.5, 1.0`.
<br/>For high throughput
(i.e. keep threads busy), set to for example `1.0, 4.0`. Currently hearbeats are not updated for picked executions
(i.e. keep threads busy), set to for example `1.0, 4.0`. Currently heartbeats are not updated for picked executions
in queue (applicable if `upperLimitFractionOfThreads > 1.0`). If they stay there for more than
`4 * heartbeat-interval` (default `20m`), not starting execution, they will be detected as _dead_ and likely be
unlocked again (determined by `DeadExecutionHandler`). Currently supported by PostgreSQL, SQL Server, MySQL v8+.
@@ -333,7 +333,7 @@ but db-scheduler also bundles a `GsonSerializer` and `JacksonSerializer`. See ex
See also additional documentation under [Serializers](#Serializers).

:gear: `.executorService(ExecutorService)`<br/>
If specified, use this externally managed executor service to run executions. Ideally the number of threads it
If specified, use this externally managed executor service to run executions. Ideally, the number of threads it
will use should still be supplied (for scheduler polling optimizations). Default `null`.

:gear: `.deleteUnresolvedAfter(Duration)`<br/>
@@ -395,7 +395,7 @@ and it will remove any existing executions for that task.
### Serializers

A task-instance may have some associated data in the field `task_data`. The scheduler uses a `Serializer` to read and write this
data to the database. By default, standard Java serialization is used, but a number of options is provided:
data to the database. By default, standard Java serialization is used, but a number of options are provided:

* `GsonSerializer`
* `JacksonSerializer`
@@ -453,6 +453,7 @@ Configuration is mainly done via `application.properties`. Configuration of sche

db-scheduler.enabled=true
db-scheduler.heartbeat-interval=5m
db-scheduler.missed-heartbeats-limit=6
db-scheduler.polling-interval=10s
db-scheduler.polling-strategy=fetch
db-scheduler.polling-strategy-lower-limit-fraction-of-threads=0.5
@@ -553,7 +554,7 @@ Use-cases might be:

During execution, the scheduler regularly updates a heartbeat-time for the task-execution.
If an execution is marked as executing, but is not receiving updates to the heartbeat-time,
it will be considered a _dead execution_ after time X. That may for example happen if the
it will be considered a _dead execution_ after time X. That may, for example, happen if the
JVM running the scheduler suddenly exits.

When a dead execution is found, the `Task`is consulted to see what should be done. A dead
@@ -580,9 +581,9 @@ In v10, a new polling strategy (`lock-and-fetch`) was added. It utilizes the fac
Using such a strategy, it is possible to fetch executions pre-locked, and thus getting one statement less:

1. `select for update .. skip locked` a batch of due executions. These will already be picked by the scheduler-instance.
3. When execution is done, `update` or `delete` the record according to handlers.
2. When execution is done, `update` or `delete` the record according to handlers.

In sum per batch: 1 select-and-update, 1 * batch-size updates (no misses)
In sum per batch: 1 select-and-update, 1 * batch-size updates (no misses)


### Benchmark test
Original file line number Diff line number Diff line change
@@ -141,6 +141,8 @@ public Scheduler scheduler(DbSchedulerCustomizer customizer, StatsRegistry regis

builder.heartbeatInterval(config.getHeartbeatInterval());

builder.missedHeartbeatsLimit(config.getMissedHeartbeatsLimit());

// Use scheduler name implementation from customizer if available, otherwise use
// configured scheduler name (String). If both is absent, use the library default
if (customizer.schedulerName().isPresent()) {
Original file line number Diff line number Diff line change
@@ -37,6 +37,13 @@ public class DbSchedulerProperties {
@DurationUnit(MINUTES)
private Duration heartbeatInterval = SchedulerBuilder.DEFAULT_HEARTBEAT_INTERVAL;

/**
* How many heart beats can be missed before an execution is considered dead.
*
* <p>Must be greater than or equal to 4
*/
private int missedHeartbeatsLimit = SchedulerBuilder.DEFAULT_MISSED_HEARTBEATS_LIMIT;

/**
* Name of this scheduler-instance. The name is stored in the database when an execution is picked
* by a scheduler.
@@ -141,6 +148,14 @@ public void setHeartbeatInterval(final Duration heartbeatInterval) {
this.heartbeatInterval = heartbeatInterval;
}

public int getMissedHeartbeatsLimit() {
return missedHeartbeatsLimit;
}

public void setMissedHeartbeatsLimit(final int missedHeartbeatsLimit) {
this.missedHeartbeatsLimit = missedHeartbeatsLimit;
}

public String getSchedulerName() {
return schedulerName;
}
Original file line number Diff line number Diff line change
@@ -2,6 +2,7 @@

import static com.github.kagkarlsson.scheduler.SchedulerBuilder.DEFAULT_DELETION_OF_UNRESOLVED_TASKS_DURATION;
import static com.github.kagkarlsson.scheduler.SchedulerBuilder.DEFAULT_HEARTBEAT_INTERVAL;
import static com.github.kagkarlsson.scheduler.SchedulerBuilder.DEFAULT_MISSED_HEARTBEATS_LIMIT;
import static com.github.kagkarlsson.scheduler.SchedulerBuilder.DEFAULT_POLLING_INTERVAL;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.fail;
@@ -89,6 +90,7 @@ public void it_should_use_the_default_values_from_library() {
DbSchedulerProperties props = ctx.getBean(DbSchedulerProperties.class);
assertThat(props.getPollingInterval()).isEqualTo(DEFAULT_POLLING_INTERVAL);
assertThat(props.getHeartbeatInterval()).isEqualTo(DEFAULT_HEARTBEAT_INTERVAL);
assertThat(props.getMissedHeartbeatsLimit()).isEqualTo(DEFAULT_MISSED_HEARTBEATS_LIMIT);
assertThat(props.getDeleteUnresolvedAfter())
.isEqualTo(DEFAULT_DELETION_OF_UNRESOLVED_TASKS_DURATION);
});