Error An HTTP Client raised and unhandled exception: 's3' #173

ambrosiozack · 2018-11-22T15:30:18Z

Issue Details

Hello
When trying to make the command "s4 sync / folder" I get the following error:

Any idea? Thanks

Operating System (uname -a)

Linux debian-dhcp 4.9.0-6-amd64 #1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07) x86_64 GNU/Linux

s4 version (s4 version)

0.4.1

python version (python --version)

Python 2.7.13

Steps to reproduce the issue

When executing the command "s4 sync socios"

DEBUG log output (run with `s4 --log-level=DEBUG`)

INFO:sync_command:102 Syncing socios [/socios/ <=> s3://pixelcompruebaprivado2/]
DEBUG:local:70 Locking /socios/.s4lock
DEBUG:sync:84 Generating deferred calls based on client states
DEBUG:local:36 Ignoring <DirEntry '.s4lock'>
DEBUG:local:81 Releasing lock /socios/.s4lock
ERROR:sync_command:109 An HTTP Client raised and unhandled exception: 's3'
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/botocore/httpsession.py", line 246, in send
    conn = manager.connection_from_url(request.url)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/poolmanager.py", line 280, in connection_from_url
    pool_kwargs=pool_kwargs)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/poolmanager.py", line 228, in connection_from_host
    return self.connection_from_context(request_context)
  File "/usr/local/lib/python3.5/dist-packages/urllib3/poolmanager.py", line 238, in connection_from_context
    pool_key_constructor = self.key_fn_by_scheme[scheme]
KeyError: 's3'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/s4/commands/sync_command.py", line 105, in run
    conflict_choice=self.args.conflicts, dry_run=self.args.dry_run
  File "/usr/local/lib/python3.5/dist-packages/s4/sync.py", line 39, in sync
    resolutions, unhandled_events = self.get_sync_states(keys)
  File "/usr/local/lib/python3.5/dist-packages/s4/sync.py", line 85, in get_sync_states
    for key, state_1, state_2 in self.get_states(keys):
  File "/usr/local/lib/python3.5/dist-packages/s4/sync.py", line 321, in get_states
    client_2_actions = self.client_2.get_all_actions()
  File "/usr/local/lib/python3.5/dist-packages/s4/clients/__init__.py", line 204, in get_all_actions
    real_local_timestamps = self.get_all_real_local_timestamps()
  File "/usr/local/lib/python3.5/dist-packages/s4/clients/s3.py", line 222, in get_all_real_local_timestamps
    for page in page_iterator:
  File "/usr/local/lib/python3.5/dist-packages/botocore/paginate.py", line 255, in __iter__
    response = self._make_request(current_kwargs)
  File "/usr/local/lib/python3.5/dist-packages/botocore/paginate.py", line 332, in _make_request
    return self._method(**current_kwargs)
  File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 320, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.5/dist-packages/botocore/client.py", line 611, in _make_api_call
    operation_model, request_dict)
  File "/usr/local/lib/python3.5/dist-packages/botocore/endpoint.py", line 102, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/local/lib/python3.5/dist-packages/botocore/endpoint.py", line 136, in _send_request
    success_response, exception):
  File "/usr/local/lib/python3.5/dist-packages/botocore/endpoint.py", line 210, in _needs_retry
    caught_exception=caught_exception, request_dict=request_dict)
  File "/usr/local/lib/python3.5/dist-packages/botocore/hooks.py", line 356, in emit
    return self._emitter.emit(aliased_event_name, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/botocore/hooks.py", line 228, in emit
    return self._emit(event_name, kwargs)
  File "/usr/local/lib/python3.5/dist-packages/botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "/usr/local/lib/python3.5/dist-packages/botocore/retryhandler.py", line 183, in __call__
    if self._checker(attempts, response, caught_exception):
  File "/usr/local/lib/python3.5/dist-packages/botocore/retryhandler.py", line 251, in __call__
    caught_exception)
  File "/usr/local/lib/python3.5/dist-packages/botocore/retryhandler.py", line 269, in _should_retry
    return self._checker(attempt_number, response, caught_exception)
  File "/usr/local/lib/python3.5/dist-packages/botocore/retryhandler.py", line 317, in __call__
    caught_exception)
  File "/usr/local/lib/python3.5/dist-packages/botocore/retryhandler.py", line 223, in __call__
    attempt_number, caught_exception)
  File "/usr/local/lib/python3.5/dist-packages/botocore/retryhandler.py", line 359, in _check_caught_exception
    raise caught_exception
  File "/usr/local/lib/python3.5/dist-packages/botocore/endpoint.py", line 179, in _get_response
    http_response = self._send(request)
  File "/usr/local/lib/python3.5/dist-packages/botocore/endpoint.py", line 223, in _send
    return self.http_session.send(request)
  File "/usr/local/lib/python3.5/dist-packages/botocore/httpsession.py", line 294, in send
    raise HTTPClientError(error=e)
botocore.exceptions.HTTPClientError: An HTTP Client raised and unhandled exception: 's3'

The text was updated successfully, but these errors were encountered:

ambrosiozack · 2018-11-22T15:44:24Z

Python 3.5 is installed:

ambrosiozack · 2018-11-22T17:20:26Z

I changed python version to 3.5, but it still does not work:

MichaelAquilina · 2018-11-26T10:58:15Z

Sorry for the delay @ambrosiozack

That is quite an odd bug. Seems like something weird is going on in botocore. Could you give me the version of botocore you have on your system? (you can paste the output of pip freeze if you wish).

ambrosiozack · 2018-11-30T12:06:07Z

This is the result:

root@debian-dhcp:/home/pixelcom# pip3 freeze
args==0.1.0
boto3==1.9.57
botocore==1.12.57
chardet==2.3.0
clint==0.5.1
cryptography==1.7.1
docutils==0.14
filelock==3.0.10
httplib2==0.9.2
idna==2.2
inotify-simple==1.1.8
jmespath==0.9.3
keyring==10.1
keyrings.alt==1.3
pathspec==0.5.9
pyasn1==0.1.9
pycrypto==2.6.1
pycurl==7.43.0
pygobject==3.22.0
python-apt==1.4.0b3
python-dateutil==2.7.5
python-debian==0.1.30
python-debianbts==2.6.1
python-magic==0.4.15
pyxdg==0.25
reportbug==7.1.7
requests==2.12.4
s3transfer==0.1.13
s4==0.4.1
SecretStorage==2.3.1
six==1.10.0
tabulate==0.8.2
tqdm==4.28.1
urllib3==1.24.1

MichaelAquilina · 2018-11-30T12:09:05Z

these seem to work for me.

Could you try running pip install --upgrade boto3?

ambrosiozack · 2018-11-30T14:15:24Z

This is result:

MichaelAquilina · 2018-11-30T14:17:49Z

alright lets see what we can find by running it with pdb.

try running the following command

python3 -m pdb "$(which s4)" sync

That will drop you into a pdb shell which will hit a breakpoint as soon as the error hits. Once you're there. Could you try investigate the state of variables and see what stands out as being odd?

Let me know if you are not familiar with python debugging and need some help with this.

ambrosiozack · 2018-11-30T14:25:18Z

When I execute the command, it shows the following in the console:

Excuse me, I'm not at all familiar with Python :(

MichaelAquilina · 2018-11-30T14:26:49Z

Type c and then press enter. The shell will continue.

It will then stop at the point that the error occurs. When you get to that point enter ll and paste what you see.

ambrosiozack · 2018-11-30T14:31:06Z

This result:

ambrosiozack · 2018-11-30T14:31:56Z

MichaelAquilina · 2018-11-30T14:31:56Z

oh right sorry about that - errors are being suppressed atm so this is actually harder than I expected. I'll think of something and get back to you :)

ambrosiozack · 2018-11-30T14:33:37Z

I put more information, I had read wrong and interpreted "11" instead of "ll"

MichaelAquilina · 2018-11-30T14:34:26Z

I saw that - the issue is that pdb is not stopping at the point we need it to because the code is suppressing the errors. I'll need to take a look at the code to see how we can get it to work as we intended.

ambrosiozack · 2018-11-30T14:35:35Z

OK thanks for the help. I wait your answer.

ambrosiozack · 2018-12-13T11:19:52Z

Hi friend
Is there any progress regarding the error?
A greeting

MichaelAquilina · 2018-12-13T22:51:03Z

sorry about that @ambrosiozack. Its been a busy few weeks for me.

Could you perhaps paste the contents of ~/.config/s4

Please make sure to remove your keys before pasting to github

ambrosiozack · 2019-01-16T08:44:06Z

Hello,

This is the content to sync.conf

I try delete the first s3:// of s3_uri but it still does not work

MichaelAquilina · 2019-01-16T10:23:46Z

A few things:

The s3_uri looks incorrect. Could you try editing the file with vim to instead be:
s3://pixelcompruebaprivado2
your local folder looks strange. What is the location you intend to synchronise?
While you did remove some of the text from the secret access key I'm sure a persistent attacker would be able to guess the value because you left quite a lot of hints about each character at the top. I would recommend you rotate your secret access key to prevent anyone doing bad things with your AWS account

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error An HTTP Client raised and unhandled exception: 's3' #173

Error An HTTP Client raised and unhandled exception: 's3' #173

ambrosiozack commented Nov 22, 2018 •

edited by MichaelAquilina

Loading

ambrosiozack commented Nov 22, 2018

ambrosiozack commented Nov 22, 2018

MichaelAquilina commented Nov 26, 2018

ambrosiozack commented Nov 30, 2018 •

edited by MichaelAquilina

Loading

MichaelAquilina commented Nov 30, 2018

ambrosiozack commented Nov 30, 2018

MichaelAquilina commented Nov 30, 2018 •

edited

Loading

ambrosiozack commented Nov 30, 2018

MichaelAquilina commented Nov 30, 2018

ambrosiozack commented Nov 30, 2018

ambrosiozack commented Nov 30, 2018

MichaelAquilina commented Nov 30, 2018

ambrosiozack commented Nov 30, 2018

MichaelAquilina commented Nov 30, 2018

ambrosiozack commented Nov 30, 2018

ambrosiozack commented Dec 13, 2018

MichaelAquilina commented Dec 13, 2018 •

edited

Loading

ambrosiozack commented Jan 16, 2019

MichaelAquilina commented Jan 16, 2019 •

edited

Loading

Error An HTTP Client raised and unhandled exception: 's3' #173

Error An HTTP Client raised and unhandled exception: 's3' #173

Comments

ambrosiozack commented Nov 22, 2018 • edited by MichaelAquilina Loading

Issue Details

Operating System (uname -a)

s4 version (s4 version)

python version (python --version)

Steps to reproduce the issue

DEBUG log output (run with s4 --log-level=DEBUG)

ambrosiozack commented Nov 22, 2018

ambrosiozack commented Nov 22, 2018

MichaelAquilina commented Nov 26, 2018

ambrosiozack commented Nov 30, 2018 • edited by MichaelAquilina Loading

MichaelAquilina commented Nov 30, 2018

ambrosiozack commented Nov 30, 2018

MichaelAquilina commented Nov 30, 2018 • edited Loading

ambrosiozack commented Nov 30, 2018

MichaelAquilina commented Nov 30, 2018

ambrosiozack commented Nov 30, 2018

ambrosiozack commented Nov 30, 2018

MichaelAquilina commented Nov 30, 2018

ambrosiozack commented Nov 30, 2018

MichaelAquilina commented Nov 30, 2018

ambrosiozack commented Nov 30, 2018

ambrosiozack commented Dec 13, 2018

MichaelAquilina commented Dec 13, 2018 • edited Loading

ambrosiozack commented Jan 16, 2019

MichaelAquilina commented Jan 16, 2019 • edited Loading

ambrosiozack commented Nov 22, 2018 •

edited by MichaelAquilina

Loading

DEBUG log output (run with `s4 --log-level=DEBUG`)

ambrosiozack commented Nov 30, 2018 •

edited by MichaelAquilina

Loading

MichaelAquilina commented Nov 30, 2018 •

edited

Loading

MichaelAquilina commented Dec 13, 2018 •

edited

Loading

MichaelAquilina commented Jan 16, 2019 •

edited

Loading