blallo
/
BotZ
1
0
Fork 0
Code Issues 1 Pull Requests Releases 3 Wiki Activity

Cleanup

master
blallo 2019-01-14 17:51:19 +01:00 committed by blallo
parent c4ac8f96f9
commit dc20c91933
Signed by: blallo
GPG Key ID: 0CBE577C9B72DC3F
8 changed files with 282 additions and 188 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.gitignore vendored
View File

@ -1,3 +1,6 @@
# Postinstall bineries
/bot_z/bin/*
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]

114
CONTRIBUTING.rst
View File

@ -1,114 +0,0 @@
.. highlight:: shell
============
Contributing
============
Contributions are welcome, and they are greatly appreciated! Every
little bit helps, and credit will always be given.
You can contribute in many ways:
Types of Contributions
----------------------
Report Bugs
~~~~~~~~~~~
Report bugs at https://github.com/lbarcaroli/bot_z/issues.
If you are reporting a bug, please include:
* Your operating system name and version.
* Any details about your local setup that might be helpful in troubleshooting.
* Detailed steps to reproduce the bug.
Fix Bugs
~~~~~~~~
Look through the GitHub issues for bugs. Anything tagged with "bug"
and "help wanted" is open to whoever wants to implement it.
Implement Features
~~~~~~~~~~~~~~~~~~
Look through the GitHub issues for features. Anything tagged with "enhancement"
and "help wanted" is open to whoever wants to implement it.
Write Documentation
~~~~~~~~~~~~~~~~~~~
Bot_Z could always use more documentation, whether as part of the
official Bot_Z docs, in docstrings, or even on the web in blog posts,
articles, and such.
Submit Feedback
~~~~~~~~~~~~~~~
The best way to send feedback is to file an issue at https://github.com/lbarcaroli/bot_z/issues.
If you are proposing a feature:
* Explain in detail how it would work.
* Keep the scope as narrow as possible, to make it easier to implement.
* Remember that this is a volunteer-driven project, and that contributions
are welcome :)
Get Started!
------------
Ready to contribute? Here's how to set up `bot_z` for local development.
1. Fork the `bot_z` repo on GitHub.
2. Clone your fork locally::
$ git clone git@github.com:your_name_here/bot_z.git
3. Install your local copy into a virtualenv. Assuming you have virtualenvwrapper installed, this is how you set up your fork for local development::
$ mkvirtualenv bot_z
$ cd bot_z/
$ python setup.py develop
4. Create a branch for local development::
$ git checkout -b name-of-your-bugfix-or-feature
Now you can make your changes locally.
5. When you're done making changes, check that your changes pass flake8 and the tests, including testing other Python versions with tox::
$ flake8 bot_z tests
$ python setup.py test or py.test
$ tox
To get flake8 and tox, just pip install them into your virtualenv.
6. Commit your changes and push your branch to GitHub::
$ git add .
$ git commit -m "Your detailed description of your changes."
$ git push origin name-of-your-bugfix-or-feature
7. Submit a pull request through the GitHub website.
Pull Request Guidelines
-----------------------
Before you submit a pull request, check that it meets these guidelines:
1. The pull request should include tests.
2. If the pull request adds functionality, the docs should be updated. Put
your new functionality into a function with a docstring, and add the
feature to the list in README.rst.
3. The pull request should work for Python 2.6, 2.7, 3.3, 3.4 and 3.5, and for PyPy. Check
https://travis-ci.org/lbarcaroli/bot_z/pull_requests
and make sure that the tests pass for all supported Python versions.
Tips
----
To run a subset of tests::
$ python -m unittest tests.test_bot_z

8
HISTORY.rst
View File

@ -1,8 +0,0 @@
=======
History
=======
0.1.0 (2019-01-07)
------------------
* First release on PyPI.

2
MANIFEST.in
View File

@ -1,5 +1,3 @@
include CONTRIBUTING.rst
include HISTORY.rst
include LICENSE
include README.rst

90
bot_z/bot_z.py
View File

@ -8,25 +8,28 @@ service. It exposes methods to login, logout and check in and out.
from datetime import datetime, timedelta
import logging
import os
import pkg_resources
import time
import typing as T
from urllib.parse import urlparse
os.environ['PATH'] = os.environ['PATH'] + \
':' + os.path.join(os.path.abspath(os.path.curdir), 'bin')
os.environ["PATH"] = (
os.environ["PATH"] + ":" + pkg_resources.resource_filename(__name__, "bin")
)
from selenium import webdriver as wd
from selenium.common.exceptions import WebDriverException, NoSuchElementException
logging.basicConfig(
level=os.environ.get('BOTZ_LOGLEVEL', logging.INFO),
format='%(levelname)s: [%(name)s] -> %(message)s'
level=os.environ.get("BOTZ_LOGLEVEL", logging.INFO),
format="%(levelname)s: [%(name)s] -> %(message)s",
)
m_logger = logging.getLogger(__name__)
m_logger.debug("Init at debug")
def safely(f: T.Callable) -> T.Callable:
def _protection(self, *args, **kwargs):
try:
@ -47,10 +50,9 @@ def _is_present(driver: wd.Firefox, xpath: str) -> bool:
return False
def is_present(driver: wd.Firefox,
xpath: str,
timeout: T.Optional[timedelta]=None
) -> bool:
def is_present(
driver: wd.Firefox, xpath: str, timeout: T.Optional[timedelta] = None
) -> bool:
"""
Helper function. If an element is present in the DOM tree,
returns true. False otherwise.
@ -72,22 +74,24 @@ def is_present(driver: wd.Firefox,
class Operator(wd.Firefox):
def __init__(
self,
base_uri: str,
name: str = None,
timeout: int=20,
proxy: T.Optional[T.Tuple[str, int]] = None,
headless: bool = True,
debug: bool = False,
*args, **kwargs) -> None:
self,
base_uri: str,
name: str = None,
timeout: int = 20,
proxy: T.Optional[T.Tuple[str, int]] = None,
headless: bool = True,
debug: bool = False,
*args,
**kwargs
) -> None:
"""
Adds some configuration to Firefox.
"""
self.profile = wd.FirefoxProfile()
# Do not send telemetry
self.profile.set_preference('datareporting.policy.dataSubmissionEnabled', False)
self.profile.set_preference('datareporting.healthreport.service.enabled', False)
self.profile.set_preference('datareporting.healthreport.uploadEnabled', False)
self.profile.set_preference("datareporting.policy.dataSubmissionEnabled", False)
self.profile.set_preference("datareporting.healthreport.service.enabled", False)
self.profile.set_preference("datareporting.healthreport.uploadEnabled", False)
self.opts = wd.firefox.options.Options()
self.opts.headless = headless
@ -98,13 +102,15 @@ class Operator(wd.Firefox):
self.timeout = timedelta(seconds=timeout)
if proxy:
self.profile.set_preference('network.proxy.type', 1)
self.profile.set_preference('network.proxy.http', proxy[0])
self.profile.set_preference('network.proxy.http_port', proxy[1])
self.profile.set_preference('network.proxy.ssl', proxy[0])
self.profile.set_preference('network.proxy.ssl_port', proxy[1])
self.profile.set_preference("network.proxy.type", 1)
self.profile.set_preference("network.proxy.http", proxy[0])
self.profile.set_preference("network.proxy.http_port", proxy[1])
self.profile.set_preference("network.proxy.ssl", proxy[0])
self.profile.set_preference("network.proxy.ssl_port", proxy[1])
super().__init__(firefox_profile=self.profile, options=self.opts, *args, **kwargs)
super().__init__(
firefox_profile=self.profile, options=self.opts, *args, **kwargs
)
self.fullscreen_window()
self.z_name = name if name is not None else __name__
@ -116,7 +122,7 @@ class Operator(wd.Firefox):
self._checked_in = False
@safely
def login(self, user: str, password: str, force: bool=False) -> None:
def login(self, user: str, password: str, force: bool = False) -> None:
"""
Do the login and proceed.
"""
@ -127,22 +133,22 @@ class Operator(wd.Firefox):
self.logger.info("Forcing login: %s", user)
# Retrieve login page
self.get(self.base_uri)
_correct_url = 'cpccchk' in self.current_url
_correct_url = "cpccchk" in self.current_url
_now = datetime.now()
_elapsed = timedelta(seconds=0)
while not _correct_url:
self.logger.debug("Not yet on login page: %s", self.current_url)
time.sleep(0.5)
_correct_url = 'cpccchk' in self.current_url
_correct_url = "cpccchk" in self.current_url
_elapsed = datetime.now() - _now
if _elapsed > self.timeout:
break
self.logger.debug("After login get: %s", self.current_url)
time.sleep(1)
# Username
user_form = self.find_element_by_name('m_cUserName')
user_form = self.find_element_by_name("m_cUserName")
# Password
pass_form = self.find_element_by_name('m_cPassword')
pass_form = self.find_element_by_name("m_cPassword")
# Login button
login_butt = self.find_element_by_xpath('//input[contains(@id, "_Accedi")]')
# Compile and submit
@ -156,7 +162,7 @@ class Operator(wd.Firefox):
login_butt.submit()
time.sleep(5)
self.logger.debug("Login result: %s", self.title)
if 'Routine window' in self.title:
if "Routine window" in self.title:
self.logger.debug("Reloading...")
self.refresh()
self.switch_to.alert.accept()
@ -167,7 +173,7 @@ class Operator(wd.Firefox):
self.logger.error("Login failed: %s", user)
@safely
def logout(self, user: str, force: bool=False) -> None:
def logout(self, user: str, force: bool = False) -> None:
"""
Do the logout.
"""
@ -177,7 +183,9 @@ class Operator(wd.Firefox):
return
self.logger.info("Forcing logout: %s", user)
# Find the Profile menu and open it
profile_butt = self.find_element_by_xpath('//span[contains(@id, "imgNoPhotoLabel")]')
profile_butt = self.find_element_by_xpath(
'//span[contains(@id, "imgNoPhotoLabel")]'
)
profile_butt.click()
time.sleep(1)
# Find the logout button
@ -194,14 +202,14 @@ class Operator(wd.Firefox):
Check if already logged in. Checks if page is '/jsp/home.jsp'
and if login cookie is set (and not expired).
"""
_base_domain = '.'.join(self.uri.netloc.split('.')[-2:])
cookies = [c['name'] for c in self.get_cookies() if _base_domain in c['domain']]
_base_domain = ".".join(self.uri.netloc.split(".")[-2:])
cookies = [c["name"] for c in self.get_cookies() if _base_domain in c["domain"]]
_right_url = "/jsp/home.jsp" in self.current_url
_cookies = "dtLatC" in cookies
return _right_url and _cookies
@safely
def check_in(self, force: bool=False) -> None:
def check_in(self, force: bool = False) -> None:
"""
Click the check in button.
"""
@ -212,7 +220,9 @@ class Operator(wd.Firefox):
self.logger.warn("Already checked in!")
if not force:
return
iframe = self.find_element_by_xpath('//iframe[contains(@id, "gsmd_container.jsp")]')
iframe = self.find_element_by_xpath(
'//iframe[contains(@id, "gsmd_container.jsp")]'
)
self.switch_to.frame(iframe)
enter_butt = self.find_element_by_xpath('//input[@value="Entrata"]')
enter_butt.click()
@ -221,7 +231,7 @@ class Operator(wd.Firefox):
pass
@safely
def check_out(self, force: bool=False) -> None:
def check_out(self, force: bool = False) -> None:
"""
Click the check out button.
"""
@ -232,7 +242,9 @@ class Operator(wd.Firefox):
self.logger.warn("Not yet checked in!")
if not force:
return
iframe = self.find_element_by_xpath('//iframe[contains(@id, "gsmd_container.jsp")]')
iframe = self.find_element_by_xpath(
'//iframe[contains(@id, "gsmd_container.jsp")]'
)
self.switch_to.frame(iframe)
exit_butt = self.find_element_by_xpath('//input[@value="Uscita"]')
exit_butt.click()

1
requirements_build.txt 100644
View File

@ -0,0 +1 @@
wheel>=0.32.3

2
setup.cfg
View File

@ -12,7 +12,7 @@ search = __version__ = '{current_version}'
replace = __version__ = '{new_version}'
[bdist_wheel]
universal = 1
universal = 0
[flake8]
exclude = docs

250
setup.py
View File

@ -1,49 +1,251 @@
#!/usr/bin/env python
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""The setup script."""
from setuptools import setup, find_packages
from collections import namedtuple
from html.parser import HTMLParser
import os
import pkg_resources
from setuptools import setup, find_packages # noqa
from setuptools.command.develop import develop # noqa
from setuptools.command.install import install # noqa
from setuptools.command.bdist_egg import bdist_egg # noqa
import sys
import tarfile
import typing as T
from urllib.error import HTTPError, URLError
from urllib.request import urlopen
from wheel.bdist_wheel import bdist_wheel # noqa
import zipfile
with open("README.rst") as readme_file:
GECKO_RELEASE_PATH = "https://github.com/mozilla/geckodriver"
PKG_NAME = "bot_z"
VERSION = "0.1.0"
AUTHOR = "blallo"
AUTHOR_EMAIL = "blallo@autistici.org"
with open("README.md") as readme_file:
readme = readme_file.read()
with open("HISTORY.rst") as history_file:
history = history_file.read()
requirements = ["Click>=6.0", "selenium>=3.141.0"]
requirements = [
"Click>=6.0",
"selenium>=3.141.0",
# TODO: put package requirements here
]
setup_requirements = [] # type: T.List[str]
setup_requirements = [
# TODO(lbarcaroli): put setup requirements (distutils extensions, etc.) here
]
test_requirements = [] # type: T.List[str]
class GitTags(HTMLParser):
tags: T.List[str] = list()
take_next = 0
def handle_starttag(self, tag, attrs):
dattrs = dict(attrs)
if "commit-title" in dattrs.get("class", ""):
self.take_next = 1
def handle_data(self, data):
if self.take_next == 0:
return
elif self.take_next == 1:
self.take_next = 2
elif self.take_next == 2:
self.tags.append(data.strip("\n").strip(" ").strip("\n"))
self.take_next = 0
def retrieve_page(url: str) -> bytes:
"""
Auxiliary function to download html body
from and URI, handling the errors.
"""
try:
with urlopen(url) as conn:
content = conn.read()
except HTTPError as e:
print("Connection error: {!s}".format(e))
raise
except URLError as e:
print("Check the URI: {!s}".format(e))
raise
return content
def find_latest_version(url: str) -> str:
"""
Retrieves latest geckodriver tag.
"""
tag_page = retrieve_page("{}/tags".format(url))
gt = GitTags()
gt.feed(tag_page.decode("utf-8"))
gt.tags.sort()
return gt.tags[-1]
def verify_if_superuser() -> bool:
"""
Checks if uid or euid is 0.
"""
_uid = os.getuid()
_euid = os.geteuid()
return _uid == 0 or _euid == 0
def create_local_folder() -> None:
"""
Create a bin/ folder in the current package installation path.
"""
bin_path = pkg_resources.resource_filename(PKG_NAME, BIN_PATH)
def assemble_driver_uri(
version: T.Optional[str] = None, platform: T.Optional[str] = None
) -> str:
"""
Selects the right geckodriver URI.
"""
# TODO: use pkg_resources.get_platform()
if not version:
version = find_latest_version(GECKO_RELEASE_PATH)
if not platform:
s_platform = sys.platform
is_64bits = sys.maxsize > 2 ** 32
if is_64bits:
platform = "{}64".format(s_platform)
else:
platform = "{}64".format(s_platform)
if "win" in platform:
ext = "zip"
else:
ext = "tar.gz"
return "{base}/releases/download/{vers}/geckodriver-{vers}-{platform}.{ext}".format(
base=GECKO_RELEASE_PATH, vers=version, platform=platform, ext=ext
)
def download_driver_bin(uri: str, path: str) -> None:
"""
Donwloads the geckodriver binary.
"""
name = uri.split("/")[-1]
filepath = os.path.join(path, name)
print("[DRIVER] downloading '{}' to {}".format(uri, filepath))
content = retrieve_page(uri)
try:
with open(filepath, "wb") as f:
f.write(content)
if name.endswith(".zip"):
with zipfile.ZipFile(filepath, "r") as z:
z.extractall(path)
elif name.endswith(".tar.gz"):
with tarfile.open(filepath, "r") as r:
r.extractall(path)
else:
raise RuntimeError("Unrecognized file extension: %s", name)
finally:
os.remove(filepath)
def postinstall(platform: T.Optional[str] = None) -> None:
"""
Performs all the postintallation flow, donwloading in the
right place the geckodriver binary.
"""
# target_path = os.path.join(os.path.abspath(os.path.curdir), 'bot_z', 'bin')
target_path = pkg_resources.resource_filename("bot_z", "bin")
pkg_resources.ensure_directory(os.path.join(target_path, "target"))
version = os.environ.get("BOTZ_GECKO_VERSION")
gecko_uri = assemble_driver_uri(version, platform)
print("[POSTINSTALL] gecko_uri: {}".format(gecko_uri))
download_driver_bin(gecko_uri, target_path)
def translate_platform_to_gecko_vers(plat: str) -> str:
"""
Map appropriately the platform provided on the command line
to the one used by PEP 513.
"""
PLATS = {
"win32": "win32",
"win-amd64": "win64",
"manylinux1-i686": "linux32",
"manylinux1-x86_64": "linux64",
"macosx": "macos",
}
try:
return PLATS[plat]
except KeyError as e:
print("Allowed platforms are: {!r}".format(list(PLATS.keys())))
raise
# From: https://stackoverflow.com/a/36902139
class CustomDevelopCommand(develop):
"""Custom installation for development mode."""
def run(self):
super().run()
print("POSTINSTALL")
postinstall()
class CustomInstallCommand(install):
"""Custom installation for installation mode."""
def run(self):
super().run()
opts = self.distribution.get_cmdline_options()
if "bdist_wheel" in opts:
platform = translate_platform_to_gecko_vers(
opts["bdist_wheel"].get("plat-name")
)
postinstall(platform)
# From: https://stackoverflow.com/a/45150383
class CustomBDistWheel(bdist_wheel):
"""
Custom bdist_wheel command to ship the right binary
of geckodriver.
"""
def finalize_options(self):
super().finalize_options()
self.root_is_pure = False
def get_tag(self):
python, abi, plat = super().get_tag()
python, abi = "py3", "none"
return python, abi, plat
test_requirements = [
# TODO: put package test requirements here
]
setup(
name="bot_z",
version="0.1.0",
name=PKG_NAME,
version=VERSION,
description="A bot to easen the daily routine with zucchetti virtual badge.",
long_description=readme + "\n\n" + history,
author="Blallo",
author_email="blallo@autistici.org",
url="https://github.com/lbarcaroli/bot_z",
long_description=readme,
author=AUTHOR,
author_email=AUTHOR_EMAIL,
url="https://git.abbiamoundominio.org/blallo/BotZ",
packages=find_packages(include=["bot_z"]),
cmdclass={
"develop": CustomDevelopCommand,
"install": CustomInstallCommand,
"bdist_wheel": CustomBDistWheel,
},
entry_points={"console_scripts": ["bot_z=bot_z.cli:main"]},
package_data={"bot_z": ["bot_z/bin/geckodriver"]},
include_package_data=True,
install_requires=requirements,
license="GNU General Public License v3",
license="GLWTS Public Licence",
zip_safe=False,
keywords="bot_z",
classifiers=[
"Development Status :: 2 - Pre-Alpha",
"Intended Audience :: Developers",
"License :: OSI Approved :: GNU General Public License v3 (GPLv3)",
"License :: GLWTS Public Licence",
"Natural Language :: English",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.7",