refine the implementation of IndexMergeReader #14332
Labels
help wanted
Denotes an issue that needs help from a contributor. Must meet "help wanted" guidelines.
sig/execution
SIG execution
type/enhancement
The issue or PR belongs to an enhancement.
Feature Request
Is your feature request related to a problem? Please describe:
The implementation of IndexMergeReader is rough now, we can refine it further to make the code more readable and correct.
Describe the feature you'd like:
KeepOrder
forIndexMergeReaderExecutor
.IndexMerge
does not promise the output order and does not need the order of its children now, thus we do not need theKeepOrder
attribute now.issue: polish the code of IndexMergeReaderExecutor #14339
indexMergeProcessWorker.maps
, we can define this variable as a local variable inindexMergeProcessWorker.fetchLoop
.issue: polish the code of IndexMergeReaderExecutor #14339
partialWorker
andindexMergeProcessWorker
.indexMergeProcessWorker.fetchLoop
usepartialWorkerCnt--
to check whether all thepartialWorker
s finishing their work. We can use aWaitGroup
to synchronize the work between all the partial workers, and start a goroutine which waits for the WaitGroup, and close thefetchCh
when all thepartialWorker
s exit. WhenindexMergeProcessWorker
fails to fetch data fromfetchCh
, it knows all thepartialWorker
s exit. Refer thisissue: refine the IPC in indexMergeReaderExecutor #14333
Runtime stats
andfeedback
are not handled correctly duringindexMergeReaderExecutor
, we need to make them work.make RuntimeStats and feedback correct for IndexMergeReaderExecutor #14334
WithRecovery
instead of the way used to recover to make the code more readable. Refer this.issue: use
WithRecovery
in IndexMergeReaderExecutor instead ofrecover
#14335Next
is called insteadOpen
.issue: start workers in Next instead of Open for IndexMergeReaderExecutor #14336
indexMergeProcessWorker
to save the memory usage. Refer thisissue: reuse the result chk in IndexMergeReaderExecutor #14337
handle
s to theindexMergeTableScanWorker
s instead of the handles of a task which is filtered byindexMergeProcessWorker.maps
.issue: send a batch of handles in indexMergeTableScanWorker #14338
The text was updated successfully, but these errors were encountered: