Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Implement read from Athena functionality #9

Open
5 tasks
Christos-Hadjinikolis opened this issue Jun 22, 2022 · 0 comments
Open
5 tasks

feat: Implement read from Athena functionality #9

Christos-Hadjinikolis opened this issue Jun 22, 2022 · 0 comments
Assignees
Labels
enhancement New feature or request help wanted Extra attention is needed

Comments

@Christos-Hadjinikolis
Copy link
Collaborator

Christos-Hadjinikolis commented Jun 22, 2022

Context

It is required that Dynamicio can directly read dataframes from AWS Athena.

  • Further details about getting started with dynamicio can be found here

Task

  • Implement a WithAthena mixin class with a read_from_athena method that reads tables from AWS athena

Suggested Approach

Dynamicio already uses the boto3 library to access AWS. You'll want to continue using the boto3 class for authentication, and reading/writing where possible.

Please contact kit.burgess@miodo.io for instructions on how to test your code against a test AWS Athena instance.

Acceptance Criteria

  • The read_from_athena method should read whole tables (every column & row) directly from AWS athena. select * from tablename.
  • All existing tests pass.
  • New tests written as required.
  • Readme & docs are updated to reflect the new changes.

The Process

Please fork this repo, commit your changes on a branch, then send @miodohelper a PR request when your work is ready for review. Miodo will review the code before sending the PR for a final review by @Christos-Hadjinikolis.

Expected Duration

8hrs

@Christos-Hadjinikolis Christos-Hadjinikolis moved this from Backlog to Ready to be picked up in dynamicio Jul 13, 2022
@miodohelper miodohelper self-assigned this Jul 14, 2022
@miodohelper miodohelper changed the title feat: Implement a mixin for ATHENA feat: Implement a read and write ATHENA mixin Jul 14, 2022
@miodohelper miodohelper changed the title feat: Implement a read and write ATHENA mixin feat: Implement read from Athena functionality Jul 22, 2022
@Christos-Hadjinikolis Christos-Hadjinikolis added enhancement New feature or request help wanted Extra attention is needed labels Jul 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed
Projects
Status: Ready to be picked up
Development

No branches or pull requests

2 participants