Git2S3

Backup GitHub projects to AWS S3

Platform Supported

Deployments

Kick off

Recommendations

• Install python 3.10 or 3.11

• Use a dedicated virtual environment

Install Git2S3

python -m pip install git2s3

Initiate - IDE

import git2s3


if __name__ == '__main__':
    git = git2s3.Git2S3()
    git.start()

Initiate - CLI

git2s3 start

Use git2s3 --help for usage instructions.

Environment Variables

Sourcing environment variables from an env file

By default, Git2S3 will look for a .env file in the current working directory.
Refer samples directory for examples.

• GIT_API_URL - GitHub API endpoint. Defaults to https://api.github.com/

• GIT_OWNER - GitHub profile owner or organization name.

• GIT_TOKEN - GitHub token to get ALL repos (including private).

• GIT_IGNORE - List of repositories/gists to ignore. Defaults to []

• MAX_PER_PAGE - Max number of repos/gists to pull from a single page. Defaults to 100

• SOURCE - List of source options to back up. Defaults to [repo, gist, wiki]

• LOG - Log options to log to a file or stdout. Does not apply when custom logger is used

• DEBUG - Boolean flag to enable debug level logging. Does not apply when custom logger is used

• DRY_RUN - Boolean flag to skip upload to S3. Defaults to False

• LOCAL_STORE - Boolean flag to store the backup locally. Defaults to False

• INCOMPLETE_UPLOAD - Boolean flag to upload incomplete cloning. Defaults to False

• AWS_PROFILE_NAME - AWS profile name. Uses the CLI config value AWS_DEFAULT_PROFILE by default.

• AWS_ACCESS_KEY_ID - AWS access key ID. Uses the CLI config value AWS_ACCESS_KEY_ID by default.

• AWS_SECRET_ACCESS_KEY - AWS secret key. Uses the CLI config value AWS_SECRET_ACCESS_KEY by default.

• AWS_REGION_NAME - S3 bucket’s region. Uses the CLI config value AWS_DEFAULT_REGION by default.

• AWS_BUCKET_NAME - AWS bucket name to store the backups.

• AWS_S3_PREFIX - S3 prefix (folder like) for the backup. Defaults to github

• BOTO3_RETRY_ATTEMPTS - Number of retries for Boto3 client config. Defaults to 10

• BOTO3_RETRY_MODE - Boto3 retry configuration for S3 client. Defaults to standard

• CUT_OFF_DAYS - Cut off threshold to back up only the repos/gists that were “updated”/”pushed to”

Coding Standards

Docstring format: Google
Styling conventions: PEP 8 and isort

Release Notes

Requirement

python -m pip install gitverse

Usage

gitverse-release reverse -f release_notes.rst -t 'Release Notes'

Linting

pre-commit will ensure linting, run pytest, generate runbook & release notes, and validate hyperlinks in ALL markdown files (including Wiki pages)

Requirement

python -m pip install sphinx==5.1.1 pre-commit recommonmark

Usage

pre-commit run --all-files

Pypi Package

https://pypi.org/project/git2s3/

Runbook

https://thevickypedia.github.io/git2s3/

License & copyright

© Vignesh Rao

Licensed under the MIT License