[Bug]: Read timed out?

[cybersteve547]

scratchattach version

2.1.13

What happened?

i had started the script, left it running for a while and forgot abt it, then came back watched it for a min and this happened

Your code.

import scratchattach as sa
from collections import deque
from cyberutils import clear

start = sa.get_user("cybersteve547")
queue = deque()

def index(user):
	name = user.username
	#print("Reading " + name)
	count = user.following_count()
	#print(name + " is following " + str(count) + " people!")
	if count < 100:
		for f in user.following():
			print("Adding " + f.username)
			queue.append(f)

queue.append(start)
while True:
	clear()
	print(len(queue))
	for i in range(0, min(10, len(queue))):
		print(queue[i])
	index(queue[0])
	queue.popleft()

Traceback

32995
twistedtempokkk
miak300000
WO997
griffpatch
JloAu
codingguyA20s
ZendraGames
theCharpy
Magicalunicornsxox
griffpatch
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 451, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 446, in _make_request
    httplib_response = conn.getresponse()
                       ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 1374, in getresponse
    response.begin()
  File "/usr/lib/python3.11/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
                              ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/socket.py", line 706, in readinto
    return self._sock.recv_into(b)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 1311, in recv_into
    return self.read(nbytes, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 1167, in read
    return self._sslobj.read(len, buffer)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 789, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 552, in increment
    raise six.reraise(type(error), error, _stacktrace)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/six.py", line 719, in reraise
    raise value
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 705, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 453, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 342, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='api.scratch.mit.edu', port=443): Read timed out. (read timeout=10)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cybersteve547/.local/lib/python3.11/site-packages/scratchattach/utils/requests.py", line 30, in get
    r = requests.get(url, data=data, json=json, headers=headers, cookies=cookies, params=params,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 578, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='api.scratch.mit.edu', port=443): Read timed out. (read timeout=10)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cybersteve547/Desktop/scratchindexer.py", line 24, in <module>
    index(queue[0])
  File "/home/cybersteve547/Desktop/scratchindexer.py", line 14, in index
    for f in user.following():
             ^^^^^^^^^^^^^^^^
  File "/home/cybersteve547/.local/lib/python3.11/site-packages/scratchattach/site/user.py", line 207, in following
    response = commons.api_iterative(
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cybersteve547/.local/lib/python3.11/site-packages/scratchattach/utils/commons.py", line 124, in api_iterative
    api_data = api_iterative_data(
               ^^^^^^^^^^^^^^^^^^^
  File "/home/cybersteve547/.local/lib/python3.11/site-packages/scratchattach/utils/commons.py", line 79, in api_iterative_data
    data = fetch_func(offs, max_req_limit)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cybersteve547/.local/lib/python3.11/site-packages/scratchattach/utils/commons.py", line 114, in fetch
    resp = requests.get(
           ^^^^^^^^^^^^^
  File "/home/cybersteve547/.local/lib/python3.11/site-packages/scratchattach/utils/requests.py", line 33, in get
    raise exceptions.FetchError(e)
scratchattach.utils.exceptions.FetchError: HTTPSConnectionPool(host='api.scratch.mit.edu', port=443): Read timed out. (read timeout=10)

[faretek1]

Does this occurr consistently?

this is a timeout error because a response was not received in time, rather than seeming like a bug in scratchattach.

it could be caused by bad connection or the server responding too slowly. it is the same as loading a web page and it takes forever to load and you end up with a timeout error page.

[Boss-1s]

It's probably because cloud is down again and it went down while your script was running. Otherwise if it was at startup you'd most likely get the nginx 502 error

[faretek1]

It's probably because cloud is down again and it went down while your script was running. Otherwise if it was at startup you'd most likely get the nginx 502 error

But it isn't using the cloud server?

[Boss-1s]

Doesn't it grab values from clouddata.scratch.mit.edu? Because that's the whole system, if cloud vars are down, that's just another way of saying, the whole server is down. Usually.

[faretek1]

Doesn't it grab values from clouddata.scratch.mit.edu?

neither sa.get_user nor user.following_count nor user.following nor user.username contact the scratch cloud data server. They use the html website (user.following_count) and api.scratch.mit.edu (sa.get_user and user.following). user.username doesn't perform any new operations, as it is an attribute which is set with sa.get_user

if cloud vars are down, that's just another way of saying, the whole server is down. Usually.

I..don't think so. As far as I know, the cloud data server is a separate server to api.scratch.mit.edu/the main website. You could check sa.get_health and sa.get_cloud_status (maybe this function should be renamed)