[DataGridPro] Make infinite scrolling virtual

[mak-elena]

Duplicates

• I have searched the existing issues

Latest version

• I have tested the latest version

Summary 💡

I tried to integrate the DataGrid in the applications I working on.
And I have to state, I really like the already available feature set (filterng, sorting, columns configurations, ......) and the incredible customizations possibilities.

The only thing I got in trouble with is infinite loading for server-side data.
The current implementations of infinite loading expects the new data being appended to the end of the data set.
In my cases data sets can be really huge (e.g. time series data) and without determinated total count (To know the total count I need first reach the end of the data set).

That's why this approach with data concatenation is not applicable here.
I also failed to find some appropriate workaround.
And this really prevents me from using the DataGridPro.

To make the infinite loadig workingfor my cases, I need to use virtualized data set.
This means more specifically:

1. When the grid is scrolled the size of the virtual data set can grow as log as new data can be fetched (similar to how it works already)
2. The grid specifies the range of rows expected for virtualized rendering ([firstRowIndex, lastRowIndex]).
3. This range is avaliable in the grid state of via some specific hook.
4. The grid expects data only in the given range: [firstRowIndex, lastRowIndex]
   That is what I mean under the virtualized data set. It is similar to the paging, but with floating data range.
5. As soon as the end of the data set is reached , the total count might be specified
6. (optional) It might make sence to switch the virtual scrolling mode explicitely via a dedicated option. Or, for example, scrolling='server' (I assume the behavior is relevant for sever-side data provider only)

I'm open for any suggestions and discussions.
And I would appreciate your feedback.

Examples 🌈

No response

Motivation 🔦

Our software is aimed to manage emergy systems of different levels (from an energy plant/city up to international network operators). It requires accumulation amd processing of huge amount of data (time series data from appliances, forecast, market data and much more), which has hierarchcal and time-period-depended relations and restrictions.
It is quite challenging even on the frontend side. :)

Order ID 💳 (optional)

No response

[Isarstyle]

As we had a similar usage with server side pagination the only way we got it to work was by initially filling up the grid with empty id numerated rows [{id:1},{id:2}....[id:10000] and then use the gridapi to update the rows when the data was requested from the server.
it works well and is virtualized/scrollable and you can jump to any scrollbar position which triggers then a loading for the needed data, but you need to know how many results you will expect in total, otherwise it is not possible.

[DanailH]

Hi, @mak-elena thanks for raising this. We are aware that currently there isn't a straightforward way to enable server-side infinite loading in the grid. We are aware of this tho and have planned to pick up the effort on it. You can check this for more information. https://github.com/mui-org/material-ui-x/milestone/15

Regarding your use case - maybe I missed something but even if we add support for server-side infinite loader we will still require you to provide the total rowCount. The reason for this is that based on the rowCount the grid can set the correct scroll size. Without it, there is no way of preallocating the space.

If there is no way for you to know the total rowCount upfront the other solution will be to use the onRowsScrollEnd and fetch new data every time the scroll reaches the bottom (but I guess that won't work for your case because it has to happen on the client-side and if the data set is the way it won't really be practical).

[robzawarski]

Just FYI. Maybe this is common knowledge, but if not, we have successfully integrated InfinateLoader.js from react-virtualized and can browse a 1 million entry dataset. Out-of-the-box, DataGridPro, when accessing more than 500k or so rows, each 40 pixels high, the bottom of the scroll-window ends at about record with index around 450k-ish. I believe it has something to do with the size of the virtual-scrollwindow being maxed out. It didnt make sense why the virtual window had to be the actual height when only a couple dozen entries were actually being shown. If you initially set the row height to 20 and then later the row and cell height to 40, it works, allowing you to scroll through at least 1 million records with a row height of 40.

[tcd]

@robzawarski could you provide an example of how you implemented that?