Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: make working with pandas dataframe timestamps easier #438

Closed
Anaisdg opened this issue May 16, 2022 · 7 comments · Fixed by #440
Closed

Feature Request: make working with pandas dataframe timestamps easier #438

Anaisdg opened this issue May 16, 2022 · 7 comments · Fixed by #440
Labels
enhancement New feature or request
Milestone

Comments

@Anaisdg
Copy link

Anaisdg commented May 16, 2022

Currently users have to do a bit of timestamp manipulation in order to write pandas DataFrames, for example:
image

The following features could make working pandas DataFrames easier:

  • The ability to specify the timestamp column/not set the timestamp column as an index
  • The ability to specify the timestamp type and automatic conversation (like from ISO or rfc3339 timestamps to datetime objects).
@bednar bednar added enhancement New feature or request state: in progress labels May 17, 2022
@bednar
Copy link
Contributor

bednar commented May 17, 2022

Hi @Anaisdg,

thanks for the suggestion, we'll take a look. Do you have a sample data?

Regards

@rslippert
Copy link

In order to use a wider variety of time and date formats,
there should also be a timezone parameter specified.
It would help if the InfluxDb could generate a json template like what my software uses,
it could generate this json data like it does when generating a token, see below:
{
"influx_bucket": "rslippert's Bucket",
"influx_org": "d9ae8eef6f1bd6da",
"influx_token": "E3zGsMqgMrDYthxjfT921FXhipsOfGOIOxpcfvn61SnBE4D1mdgYKF5aYefaCbroy0UpBIsOEEMZr2XM96dtFg==",
"influx_url": "https://us-east-1-1.aws.cloud2.influxdata.com",
"timezone": "EST",
"influx_version": "1.0"
}

@rslippert
Copy link

Attached is some sample data as requested:
1dSCCOPLMCOPX.csv

@bednar
Copy link
Contributor

bednar commented May 18, 2022

@rslippert thanks for sharing your data

@bednar
Copy link
Contributor

bednar commented May 24, 2022

Hi @rslippert,

with upcoming #440 you will be able to use the following code:

import pandas as pd

from influxdb_client import InfluxDBClient
from influxdb_client.client.write_api import SYNCHRONOUS

url = "http://localhost:8086"
token = "my-token"
org = "my-org"

with InfluxDBClient(url=url, token=token, org=org) as client:

    data_frame = pd.read_csv("path/to/1dSCCOPLMCOPX.csv")
    client \
        .write_api(write_options=SYNCHRONOUS) \
        .write(bucket="my-bucket",
               record=data_frame,
               data_frame_measurement_name="scco",
               data_frame_timestamp_column="Date",
               data_frame_timestamp_timezone="EST")

Regards

@bednar bednar added this to the 1.30.0 milestone May 24, 2022
@rslippert
Copy link

I’m impressed by the speed and completeness of this pandas timestamp change. Great job Influx!

@Anaisdg
Copy link
Author

Anaisdg commented May 25, 2022

Same thanks @bednar and @rhajek!!!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants