Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: IBM/sarama
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v1.41.0
Choose a base ref
...
head repository: IBM/sarama
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v1.41.1
Choose a head ref
  • 16 commits
  • 32 files changed
  • 5 contributors

Commits on Aug 21, 2023

  1. fix(proto): handle V3 member metadata and empty owned partitions

    Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
    dnwe committed Aug 21, 2023
    Copy the full SHA
    e9bd1b8 View commit details

Commits on Aug 22, 2023

  1. fix(doc): add missing doc for mock consumer

    The mock SetTopicMetadata entry stopped mid sentence, add remaining
    details.
    
    Fixes: #2374
    
    Signed-off-by: hsweif <alex93179@gmail.com>
    hsweif authored and dnwe committed Aug 22, 2023
    Copy the full SHA
    9282d75 View commit details
  2. chore(lint): bump golangci-lint and tweak config

    - bump golangci-lint to v1.54.2
    - enable a few additional gocritic checks
    - fix the association linter warnings
    
    Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
    dnwe committed Aug 22, 2023
    Copy the full SHA
    2b4ba74 View commit details

Commits on Aug 23, 2023

  1. chore(docs): add 1.41.0 to CHANGELOG.md

    Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
    dnwe committed Aug 23, 2023
    Copy the full SHA
    b1bf950 View commit details

Commits on Aug 24, 2023

  1. chore(ci): improve stale behaviour

    - pull in @main to get "continuation" behaviour from PR 1033
    - set 'ascending' to process issues in least-recently-updated order
    
    Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
    dnwe committed Aug 24, 2023
    Copy the full SHA
    bbf6ee4 View commit details
  2. chore(ci): add workflow_dispatch to stale

    This permits manual triggering
    
    Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
    dnwe committed Aug 24, 2023
    Copy the full SHA
    261043a View commit details
  3. chore(docs): remove gopkg.in link

    Using gopkg.in doesn't work for versions >1.20.1 as once a `go.mod` file
    exists, the go tooling enforces that the import path must match.
    
    Fixes #1510
    
    Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
    dnwe committed Aug 24, 2023
    Copy the full SHA
    09395f6 View commit details
  4. chore(ci): ignore .md-only changes

    No need to run the workflows if the only changes are to markdown
    documentation
    
    Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
    dnwe committed Aug 24, 2023
    Copy the full SHA
    c7e6bca View commit details
  5. chore(gh): add new style issue templates

    Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
    dnwe committed Aug 24, 2023
    Copy the full SHA
    78c7b63 View commit details
  6. fix(gh): correct issue template comments

    Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
    dnwe committed Aug 24, 2023
    Copy the full SHA
    ffaa252 View commit details

Commits on Aug 25, 2023

  1. fix: add paragraph break to fix markdown render

    Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
    dnwe committed Aug 25, 2023
    Copy the full SHA
    503ade3 View commit details

Commits on Aug 29, 2023

  1. chore(proto): doc CreateTopics/JoinGroup fields

    Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
    dnwe committed Aug 29, 2023
    Copy the full SHA
    a4eafb4 View commit details
  2. fix: make clear that error is configuration issue not server error (#…

    …2628)
    
    Fixes #1225
    
    Signed-off-by: Mark Hindess <mark.hindess@gmail.com>
    hindessm authored Aug 29, 2023

    Verified

    This commit was created on github.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    dedd86d View commit details

Commits on Aug 30, 2023

  1. fix(client): force Event Hubs to use V1_0_0_0 (#2633)

    The problem with supporting Azure Event Hubs is that it isn't Apache
    Kafka under the covers, it's an intermediate proxy that supports a
    subset of the Kafka APIs at various versions and then maps them onto
    Event Hubs protocol(s) at the backend. As as result Sarama's current
    mechanism of specifying the KAFKA_VERSION to determine what protocol
    versions to support and use doesn't really work properly with Event Hubs
    because it supports an unusual set of protocols and even defines minimum
    versions for ProduceRequest (v3) and FetchRequest (v4). For some reason
    EventHubs is very behind on FetchRequest and MetadataRequest so the max
    configuration you can use in Sarama is V1_0_0_0 and the minimum is
    V0_11_0_0.
    
    As we have recently bumped the default Version to V2_1_0_0 and support a
    wider range of protocol versions, we're more likely to see issues raised
    by Event Hubs users unless they have already pinned their version
    correctly. To try and prevent this, attempt to detect use with Event
    Hubs by inspecting the bootstrap broker addresses and overriding the
    Version.
    
    Contributes-to: #2470
    
    Signed-off-by: Dominic Evans <dominic.evans@uk.ibm.com>
    dnwe authored Aug 30, 2023

    Verified

    This commit was created on github.com and signed with GitHub’s verified signature. The key has expired.
    Copy the full SHA
    ae5eee5 View commit details
  2. fix: add retry logic to AlterUserScramCredentials

    According to KIP-554, the AlterUserScramCredentialsRequest description states:
    
      It will be will be sent to the controller and will return NOT_CONTROLLER if
      the receiving broker is not the controller.
    
    so this request should handle retries.
    
    Signed-off-by: Mark Hindess <mark.hindess@gmail.com>
    hindessm authored and dnwe committed Aug 30, 2023
    Copy the full SHA
    87229d9 View commit details
  3. fix: typo

    Signed-off-by: Mark Hindess <mark.hindess@gmail.com>
    hindessm authored and dnwe committed Aug 30, 2023
    Copy the full SHA
    f3c4194 View commit details
30 changes: 0 additions & 30 deletions .github/ISSUE_TEMPLATE.md

This file was deleted.

46 changes: 46 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
name: Bug report
about: Report an issue that you've found
title: ""
labels: ""
assignees: ""
---

##### Description

<!-- A clear and concise description of what the bug is and how to reproduce it. -->

##### Versions

<!-- Please specify real version numbers or git SHAs, not just "Latest" since that changes fairly regularly. -->

| Sarama | Kafka | Go |
| ------ | ----- | --- |
| | | |

##### Configuration

<!-- What configuration values are you using for Sarama and Kafka? -->

```go

```

##### Logs

<!-- If applicable, add logs from Sarama and/or Kafka to help explain your problem.
You can set `sarama.Logger` to a `log.Logger` to capture Sarama debug output. -->

<details><summary>logs: CLICK ME</summary>
<p>

```
```

</p>
</details>

##### Additional Context

<!-- Add any other context about the problem here. -->
16 changes: 16 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
name: Feature request
about: Suggest an idea for this project
title: ""
labels: ""
assignees: ""
---

#### Description

<!-- A clear and concise description of what the missing capability is, why it's a problem and what you want to happen -->

#### Additional context

<!-- Add any other context or links to existing implementations of the feature in other Kafka clients -->
<!-- Please link to the relevant KIP from https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals if appropriate -->
4 changes: 4 additions & 0 deletions .github/workflows/apidiff.yml
Original file line number Diff line number Diff line change
@@ -4,9 +4,13 @@ on:
push:
branches:
- main
paths-ignore:
- '**/*.md'
pull_request:
branches:
- "**"
paths-ignore:
- '**/*.md'
jobs:
apidiff:
runs-on: ubuntu-latest
6 changes: 5 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -4,9 +4,13 @@ on:
push:
branches:
- main
paths-ignore:
- '**/*.md'
pull_request:
branches:
- "**"
paths-ignore:
- '**/*.md'
jobs:
lint:
name: Linting with Go ${{ matrix.go-version }}
@@ -27,7 +31,7 @@ jobs:
GOFLAGS: -tags=functional
uses: golangci/golangci-lint-action@v3
with:
version: v1.54.0
version: v1.54.2
test:
name: Unit Testing with Go ${{ matrix.go-version }}
runs-on: ubuntu-latest
4 changes: 4 additions & 0 deletions .github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
@@ -4,9 +4,13 @@ on:
push:
branches:
- main
paths-ignore:
- '**/*.md'
pull_request:
branches:
- "**"
paths-ignore:
- '**/*.md'
schedule:
- cron: "39 12 * * 1"
jobs:
4 changes: 4 additions & 0 deletions .github/workflows/fuzz.yml
Original file line number Diff line number Diff line change
@@ -4,9 +4,13 @@ on:
push:
branches:
- main
paths-ignore:
- '**/*.md'
pull_request:
branches:
- "**"
paths-ignore:
- '**/*.md'
jobs:
test:
name: Fuzz
4 changes: 3 additions & 1 deletion .github/workflows/fvt-main.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
name: FVT
name: FVT (main)
on:
merge_group:
push:
branches:
- main
paths-ignore:
- '**/*.md'
jobs:
fvt:
name: Test with Kafka ${{ matrix.kafka-version }}
4 changes: 3 additions & 1 deletion .github/workflows/fvt-pr.yml
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
name: FVT
name: FVT (PR)
on:
pull_request:
branches:
- "**"
paths-ignore:
- '**/*.md'
jobs:
fvt:
name: Test with Kafka ${{ matrix.kafka-version }}
4 changes: 3 additions & 1 deletion .github/workflows/stale.yml
Original file line number Diff line number Diff line change
@@ -3,12 +3,14 @@ name: "Stale issues and PRs"
on:
schedule:
- cron: "0 */2 * * *"
workflow_dispatch:
jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@v8
- uses: actions/stale@main # make use of https://github.com/actions/stale/pull/1033
with:
ascending: true
days-before-stale: 90
days-before-close: 30
stale-issue-message: >-
22 changes: 8 additions & 14 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -23,18 +23,24 @@ linters-settings:
gocritic:
enabled-tags:
- diagnostic
- performance
# - experimental
# - opinionated
# - performance
# - style
enabled-checks:
- importShadow
- nestingReduce
- stringsCompare
# - unnamedResult
# - whyNoLint
disabled-checks:
- assignOp
- appendAssign
- commentedOutCode
- hugeParam
- ifElseChain
- singleCaseSwitch
- sloppyReassign
- wrapperFunc
funlen:
lines: 300
statements: 300
@@ -50,37 +56,25 @@ linters:
disable-all: true
enable:
- bodyclose
# - deadcode
- depguard
- exportloopref
- dogsled
# - dupl
- errcheck
- errorlint
- funlen
- gochecknoinits
# - goconst
- gocritic
- gocyclo
- gofmt
- goimports
# - golint
- gosec
# - gosimple
- govet
# - ineffassign
- misspell
# - nakedret
- nilerr
# - paralleltest
# - scopelint
- staticcheck
# - structcheck
# - stylecheck
- typecheck
- unconvert
- unused
# - varcheck
- whitespace

issues:
89 changes: 89 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,94 @@
# Changelog

## Version 1.41.0 (2023-08-21)

## What's Changed
### :rotating_light: Breaking Changes

Note: this version of Sarama has had a big overhaul in its adherence to the use of the right Kafka protocol versions for the given Config Version. It has also bumped the default Version set in Config (where one is not supplied) to 2.1.0. This is in preparation for Kafka 4.0 dropping support for protocol versions older than 2.1. If you are using Sarama against Kafka clusters older than v2.1.0, or using it against Azure EventHubs then you will likely have to change your application code to pin to the appropriate Version.

* chore(config): make DefaultVersion V2_0_0_0 by @dnwe in https://github.com/IBM/sarama/pull/2572
* chore(config): make DefaultVersion V2_1_0_0 by @dnwe in https://github.com/IBM/sarama/pull/2574
### :tada: New Features / Improvements
* Implement resolve_canonical_bootstrap_servers_only by @gebn in https://github.com/IBM/sarama/pull/2156
* feat: sleep when throttled (KIP-219) by @hindessm in https://github.com/IBM/sarama/pull/2536
* feat: add isValidVersion to protocol types by @dnwe in https://github.com/IBM/sarama/pull/2538
* fix(consumer): use newer LeaveGroup as appropriate by @dnwe in https://github.com/IBM/sarama/pull/2544
* Add support for up to version 4 List Groups API by @prestona in https://github.com/IBM/sarama/pull/2541
* fix(producer): use newer ProduceReq as appropriate by @dnwe in https://github.com/IBM/sarama/pull/2546
* fix(proto): ensure req+resp requiredVersion match by @dnwe in https://github.com/IBM/sarama/pull/2548
* chore(proto): permit CreatePartitionsRequest V1 by @dnwe in https://github.com/IBM/sarama/pull/2549
* chore(proto): permit AlterConfigsRequest V1 by @dnwe in https://github.com/IBM/sarama/pull/2550
* chore(proto): permit DeleteGroupsRequest V1 by @dnwe in https://github.com/IBM/sarama/pull/2551
* fix(proto): correct JoinGroup usage for wider version range by @dnwe in https://github.com/IBM/sarama/pull/2553
* fix(consumer): use full range of FetchRequest vers by @dnwe in https://github.com/IBM/sarama/pull/2554
* fix(proto): use range of OffsetCommitRequest vers by @dnwe in https://github.com/IBM/sarama/pull/2555
* fix(proto): use full range of MetadataRequest by @dnwe in https://github.com/IBM/sarama/pull/2556
* fix(proto): use fuller ranges of supported proto by @dnwe in https://github.com/IBM/sarama/pull/2558
* fix(proto): use full range of SyncGroupRequest by @dnwe in https://github.com/IBM/sarama/pull/2565
* fix(proto): use full range of ListGroupsRequest by @dnwe in https://github.com/IBM/sarama/pull/2568
* feat(proto): support for Metadata V6-V10 by @dnwe in https://github.com/IBM/sarama/pull/2566
* fix(proto): use full ranges for remaining proto by @dnwe in https://github.com/IBM/sarama/pull/2570
* feat(proto): add remaining protocol for V2.1 by @dnwe in https://github.com/IBM/sarama/pull/2573
* feat: add new error for MockDeleteTopicsResponse by @javiercri in https://github.com/IBM/sarama/pull/2475
* feat(gzip): switch to klauspost/compress gzip by @dnwe in https://github.com/IBM/sarama/pull/2600
### :bug: Fixes
* fix: correct unsupported version check by @hindessm in https://github.com/IBM/sarama/pull/2528
* fix: avoiding burning cpu if all partitions are paused by @napallday in https://github.com/IBM/sarama/pull/2532
* extend throttling metric scope by @hindessm in https://github.com/IBM/sarama/pull/2533
* Fix printing of final metrics by @prestona in https://github.com/IBM/sarama/pull/2545
* fix(consumer): cannot automatically fetch newly-added partitions unless restart by @napallday in https://github.com/IBM/sarama/pull/2563
* bug: implement unsigned modulus for partitioning with crc32 hashing by @csm8118 in https://github.com/IBM/sarama/pull/2560
* fix: avoid logging value of proxy.Dialer by @prestona in https://github.com/IBM/sarama/pull/2569
* fix(test): add missing closes to admin client tests by @dnwe in https://github.com/IBM/sarama/pull/2594
* fix(test): ensure some more clients are closed by @dnwe in https://github.com/IBM/sarama/pull/2595
* fix(examples): sync exactly_once and consumergroup by @dnwe in https://github.com/IBM/sarama/pull/2614
* fix(fvt): fresh metrics registry for each test by @dnwe in https://github.com/IBM/sarama/pull/2616
* fix(test): flaky test TestFuncOffsetManager by @napallday in https://github.com/IBM/sarama/pull/2609
### :package: Dependency updates
* chore(deps): bump the golang-org-x group with 1 update by @dependabot in https://github.com/IBM/sarama/pull/2542
* chore(deps): bump the golang-org-x group with 1 update by @dependabot in https://github.com/IBM/sarama/pull/2561
* chore(deps): bump module github.com/pierrec/lz4/v4 to v4.1.18 by @dnwe in https://github.com/IBM/sarama/pull/2589
* chore(deps): bump module github.com/jcmturner/gokrb5/v8 to v8.4.4 by @dnwe in https://github.com/IBM/sarama/pull/2587
* chore(deps): bump github.com/eapache/go-xerial-snappy digest to c322873 by @dnwe in https://github.com/IBM/sarama/pull/2586
* chore(deps): bump module github.com/klauspost/compress to v1.16.7 by @dnwe in https://github.com/IBM/sarama/pull/2588
* chore(deps): bump github.com/eapache/go-resiliency from 1.3.0 to 1.4.0 by @dependabot in https://github.com/IBM/sarama/pull/2598
### :wrench: Maintenance
* fix(fvt): ensure fully-replicated at test start by @hindessm in https://github.com/IBM/sarama/pull/2531
* chore: rollup fvt kafka to latest three by @dnwe in https://github.com/IBM/sarama/pull/2537
* Merge the two CONTRIBUTING.md's by @prestona in https://github.com/IBM/sarama/pull/2543
* fix(test): test timing error by @hindessm in https://github.com/IBM/sarama/pull/2552
* chore(ci): tidyup and improve actions workflows by @dnwe in https://github.com/IBM/sarama/pull/2557
* fix(test): shutdown MockBroker by @dnwe in https://github.com/IBM/sarama/pull/2571
* chore(proto): match HeartbeatResponse version by @dnwe in https://github.com/IBM/sarama/pull/2576
* chore(test): ensure MockBroker closed within test by @dnwe in https://github.com/IBM/sarama/pull/2575
* chore(test): ensure all mockresponses use version by @dnwe in https://github.com/IBM/sarama/pull/2578
* chore(ci): use latest Go in actions by @dnwe in https://github.com/IBM/sarama/pull/2580
* chore(test): speedup some slow tests by @dnwe in https://github.com/IBM/sarama/pull/2579
* chore(test): use modern protocol versions in FVT by @dnwe in https://github.com/IBM/sarama/pull/2581
* chore(test): fix a couple of leaks by @dnwe in https://github.com/IBM/sarama/pull/2591
* feat(fvt): experiment with per-kafka-version image by @dnwe in https://github.com/IBM/sarama/pull/2592
* chore(ci): replace toxiproxy client dep by @dnwe in https://github.com/IBM/sarama/pull/2593
* feat(fvt): add healthcheck, depends_on and --wait by @dnwe in https://github.com/IBM/sarama/pull/2601
* fix(fvt): handle msgset vs batchset by @dnwe in https://github.com/IBM/sarama/pull/2603
* fix(fvt): Metadata version in ensureFullyReplicated by @dnwe in https://github.com/IBM/sarama/pull/2612
* fix(fvt): versioned cfg for invalid topic producer by @dnwe in https://github.com/IBM/sarama/pull/2613
* chore(fvt): tweak to work across more versions by @dnwe in https://github.com/IBM/sarama/pull/2615
* feat(fvt): test wider range of kafkas by @dnwe in https://github.com/IBM/sarama/pull/2605
### :memo: Documentation
* fix(example): check if msg channel is closed by @ioanzicu in https://github.com/IBM/sarama/pull/2479
* chore: use go install for installing sarama tools by @vigith in https://github.com/IBM/sarama/pull/2599

## New Contributors
* @gebn made their first contribution in https://github.com/IBM/sarama/pull/2156
* @prestona made their first contribution in https://github.com/IBM/sarama/pull/2543
* @ioanzicu made their first contribution in https://github.com/IBM/sarama/pull/2479
* @csm8118 made their first contribution in https://github.com/IBM/sarama/pull/2560
* @javiercri made their first contribution in https://github.com/IBM/sarama/pull/2475
* @vigith made their first contribution in https://github.com/IBM/sarama/pull/2599

**Full Changelog**: https://github.com/IBM/sarama/compare/v1.40.1...v1.41.0

## Version 1.40.1 (2023-07-27)

## What's Changed
1 change: 1 addition & 0 deletions LICENSE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# MIT License

Copyright (c) 2013 Shopify

Copyright (c) 2023 IBM Corporation

Permission is hereby granted, free of charge, to any person obtaining
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -19,8 +19,9 @@ Sarama provides a "2 releases + 2 months" compatibility guarantee: we support
the two latest stable releases of Kafka and Go, and we provide a two month
grace period for older releases. However, older releases of Kafka are still likely to work.

Sarama follows semantic versioning and provides API stability via the gopkg.in service.
You can import a version with a guaranteed stable API via http://gopkg.in/IBM/sarama.v1.
Sarama follows semantic versioning and provides API stability via the standard Go
[module version numbering](https://go.dev/doc/modules/version-numbers) scheme.

A changelog is available [here](CHANGELOG.md).

## Contributing
Loading