From 56442541136f7834bc1450eb997dfdd7294b451f Mon Sep 17 00:00:00 2001 From: Tobias Genannt Date: Wed, 3 Feb 2021 20:01:36 +0100 Subject: [PATCH 01/28] Provide backends for django-storages in our images --- requirements-container.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/requirements-container.txt b/requirements-container.txt index 967c8c9..7424578 100644 --- a/requirements-container.txt +++ b/requirements-container.txt @@ -1,4 +1,4 @@ napalm==3.2.0 ruamel.yaml==0.16.12 django-auth-ldap==2.2.0 -django-storages==1.11.1 +django-storages[azure,boto3,dropbox,google,libcloud,sftp]==1.11.1 From 88b35eb48e9c9ff06fe8a067c847c995e664a334 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20Ma=CC=88der?= Date: Wed, 3 Feb 2021 21:46:11 +0100 Subject: [PATCH 02/28] Change the phrasing of the alternative options --- .github/ISSUE_TEMPLATE/config.yml | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 3ca3f71..069b882 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -1,13 +1,15 @@ blank_issues_enabled: false contact_links: - - name: The \#netbox-docker Slack channel - url: http://slack.networktocode.com/ - about: It's usually the quickest way to seek help when you're in trouble with regards to Netbox Docker. - - - name: Github Discussions + - name: Question url: https://github.com/netbox-community/netbox-docker/discussions - about: This is the right place to ask questions about how to use or do certain things with Netbox Docker. + about: The Github Discussions are the right place to ask questions about how to use or do certain things with Netbox Docker. - - name: Have you had a look at our Wiki? + - name: Chat + url: http://slack.networktocode.com/ + about: 'Usually the quickest way to seek help with small issues is to join our #netbox-docker Slack channel.' + + - name: Community Wiki url: https://github.com/netbox-community/netbox-docker/wiki - about: Our wiki contains information for common problems and tips for operating Netbox Docker in production. + about: | + Our wiki contains information for common problems and tips for operating Netbox Docker in production. + It's maintained by our excellent community. From ee40e339c856d1f68347e4102990ad95d9b2e51e Mon Sep 17 00:00:00 2001 From: Robin Schneider Date: Thu, 4 Feb 2021 21:48:08 +0100 Subject: [PATCH 03/28] NetBox should always be referred to as NetBox Fix all instances of "Netbox". ```Shell git ls-files -z . | xargs --null -I '{}' find './{}' -type f -print0 | xargs --null sed --in-place --regexp-extended 's#Netbox#NetBox#g;' ``` Ref: https://netbox.readthedocs.io/en/stable/development/style-guide/#branding --- .github/ISSUE_TEMPLATE/bug_report.md | 4 ++-- .github/ISSUE_TEMPLATE/config.yml | 4 ++-- .github/ISSUE_TEMPLATE/feature_request.md | 4 ++-- .github/workflows/push.yml | 2 +- .github/workflows/release.yml | 2 +- Dockerfile | 8 +++---- README.md | 28 +++++++++++------------ build-next.sh | 2 +- build.sh | 10 ++++---- docker/docker-entrypoint.sh | 4 ++-- test.sh | 4 ++-- 11 files changed, 36 insertions(+), 36 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 7bf0632..e11e37f 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -16,14 +16,14 @@ Before raising an issue here, answer the following questions for yourself, pleas * Have you updated to the latest version and tried again? (i.e. `git pull` and `docker-compose pull`) * Have you reset the project and tried again? (i.e. `docker-compose down -v`) * Are you confident that your problem is related to the Docker image or Docker Compose file this project provides? - (Otherwise ask on the Netbox mailing list, please: https://groups.google.com/d/forum/netbox-discuss) + (Otherwise ask on the NetBox mailing list, please: https://groups.google.com/d/forum/netbox-discuss) * Have you looked through the issues already resolved? Please try this means to get help before opening an issue here: * On the networktocode Slack in the #netbox-docker channel: http://slack.networktocode.com/ * On the networktocode Slack in the #netbox channel: http://slack.networktocode.com/ -* On the Netbox mailing list: https://groups.google.com/d/forum/netbox-discuss +* On the NetBox mailing list: https://groups.google.com/d/forum/netbox-discuss Please don't open an issue when you have a PR ready. Just submit the PR, that's good enough. diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 069b882..fe9f090 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -2,7 +2,7 @@ blank_issues_enabled: false contact_links: - name: Question url: https://github.com/netbox-community/netbox-docker/discussions - about: The Github Discussions are the right place to ask questions about how to use or do certain things with Netbox Docker. + about: The Github Discussions are the right place to ask questions about how to use or do certain things with NetBox Docker. - name: Chat url: http://slack.networktocode.com/ @@ -11,5 +11,5 @@ contact_links: - name: Community Wiki url: https://github.com/netbox-community/netbox-docker/wiki about: | - Our wiki contains information for common problems and tips for operating Netbox Docker in production. + Our wiki contains information for common problems and tips for operating NetBox Docker in production. It's maintained by our excellent community. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 6bc6dd4..c9acfcf 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -15,14 +15,14 @@ Before raising an issue here, answer the following questions for yourself, pleas * Have you had a look at the rest of the wiki? (https://github.com/netbox-community/netbox-docker/wiki) * Have you read the release notes recently (https://github.com/netbox-community/netbox-docker/releases) * Are you confident that your feature/change request is related to the Docker image or Docker Compose file this project provides? - (Otherwise ask on the Netbox mailing list, please: https://groups.google.com/d/forum/netbox-discuss) + (Otherwise ask on the NetBox mailing list, please: https://groups.google.com/d/forum/netbox-discuss) * Have you looked through the issues already resolved? Please try this means to get help before opening an issue here: * On the networktocode Slack in the #netbox-docker channel: http://slack.networktocode.com/ * On the networktocode Slack in the #netbox channel: http://slack.networktocode.com/ -* On the Netbox mailing list: https://groups.google.com/d/forum/netbox-discuss +* On the NetBox mailing list: https://groups.google.com/d/forum/netbox-discuss Please don't open an issue when you have a PR ready. Just submit the PR, that's good enough. diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml index 0615c3d..f7f9a23 100644 --- a/.github/workflows/push.yml +++ b/.github/workflows/push.yml @@ -23,7 +23,7 @@ jobs: - alpine:edge fail-fast: false runs-on: ubuntu-latest - name: Builds new Netbox Docker Images + name: Builds new NetBox Docker Images steps: - id: git-checkout name: Checkout diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 32cc9ab..1f0e317 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -18,7 +18,7 @@ jobs: - ./build.sh develop fail-fast: false runs-on: ubuntu-latest - name: Builds new Netbox Docker Images + name: Builds new NetBox Docker Images steps: - id: git-checkout name: Checkout diff --git a/Dockerfile b/Dockerfile index 272f882..3d5d5ac 100644 --- a/Dockerfile +++ b/Dockerfile @@ -81,8 +81,8 @@ LABEL ORIGINAL_TAG="" \ # Also https://microbadger.com/labels org.label-schema.schema-version="1.0" \ org.label-schema.build-date="" \ - org.label-schema.name="Netbox Docker" \ - org.label-schema.description="A container based distribution of Netbox, the free and open IPAM and DCIM solution." \ + org.label-schema.name="NetBox Docker" \ + org.label-schema.description="A container based distribution of NetBox, the free and open IPAM and DCIM solution." \ org.label-schema.vendor="The netbox-docker contributors." \ org.label-schema.url="https://github.com/netbox-community/netbox-docker" \ org.label-schema.usage="https://github.com/netbox-community/netbox-docker/wiki" \ @@ -91,8 +91,8 @@ LABEL ORIGINAL_TAG="" \ org.label-schema.version="snapshot" \ # See https://github.com/opencontainers/image-spec/blob/master/annotations.md#pre-defined-annotation-keys org.opencontainers.image.created="" \ - org.opencontainers.image.title="Netbox Docker" \ - org.opencontainers.image.description="A container based distribution of Netbox, the free and open IPAM and DCIM solution." \ + org.opencontainers.image.title="NetBox Docker" \ + org.opencontainers.image.description="A container based distribution of NetBox, the free and open IPAM and DCIM solution." \ org.opencontainers.image.licenses="Apache-2.0" \ org.opencontainers.image.authors="The netbox-docker contributors." \ org.opencontainers.image.vendor="The netbox-docker contributors." \ diff --git a/README.md b/README.md index 0125c29..16c0698 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ [![MicroBadger Size](https://img.shields.io/microbadger/image-size/netboxcommunity/netbox)][netbox-docker-microbadger] [![GitHub license](https://img.shields.io/github/license/netbox-community/netbox-docker)][netbox-docker-license] -[The Github repository](netbox-docker-github) houses the components needed to build Netbox as a Docker container. +[The Github repository](netbox-docker-github) houses the components needed to build NetBox as a Docker container. Images are built using this code and are released to [Docker Hub][netbox-dockerhub] and [Quay.io][netbox-quayio] once a day. Do you have any questions? @@ -27,14 +27,14 @@ Before opening an issue on Github, please join the [Network To Code][ntc-slack] ## Docker Tags -* `vX.Y.Z`: These are release builds, automatically built from [the corresponding releases of Netbox][netbox-releases]. -* `latest`: These are release builds, automatically built from [the `master` branch of Netbox][netbox-master]. -* `snapshot`: These are pre-release builds, automatically built from the [`develop` branch of Netbox][netbox-develop]. -* `develop-X.Y`: These are pre-release builds, automatically built from the corresponding [branch of Netbox][netbox-branches]. +* `vX.Y.Z`: These are release builds, automatically built from [the corresponding releases of NetBox][netbox-releases]. +* `latest`: These are release builds, automatically built from [the `master` branch of NetBox][netbox-master]. +* `snapshot`: These are pre-release builds, automatically built from the [`develop` branch of NetBox][netbox-develop]. +* `develop-X.Y`: These are pre-release builds, automatically built from the corresponding [branch of NetBox][netbox-branches]. Then there is currently one extra tags for each of the above tags: -* `-ldap`: Contains additional dependencies and configurations for connecting Netbox to an LDAP directory. +* `-ldap`: Contains additional dependencies and configurations for connecting NetBox to an LDAP directory. [Learn more about that in our wiki][netbox-docker-ldap]. New images are built and published automatically every ~24h. @@ -47,7 +47,7 @@ New images are built and published automatically every ~24h. ## Quickstart -To get Netbox Docker up and running run the following commands. +To get NetBox Docker up and running run the following commands. There is a more complete [_Getting Started_ guide on our wiki][wiki-getting-started] which explains every step. ```bash @@ -66,7 +66,7 @@ docker-compose up The whole application will be available after a few minutes. Open the URL `http://0.0.0.0:8000/` in a web-browser. -You should see the Netbox homepage. +You should see the NetBox homepage. In the top-right corner you can login. The default credentials are: @@ -79,7 +79,7 @@ The default credentials are: ## Documentation -Please refer [to our wiki on Github][netbox-docker-wiki] for further information on how to use this Netbox Docker image properly. +Please refer [to our wiki on Github][netbox-docker-wiki] for further information on how to use this NetBox Docker image properly. It covers advanced topics such as using files for secrets, deployment to Kubernetes, monitoring and configuring NAPALM or LDAP. [netbox-docker-wiki]: https://github.com/netbox-community/netbox-docker/wiki/ @@ -89,7 +89,7 @@ It covers advanced topics such as using files for secrets, deployment to Kuberne Feel free to ask questions in our [Github Community][netbox-community] or join [our Slack channel `#netbox-docker`][netbox-docker-slack] on the [Network To Code Slack][ntc-slack], which is free to use and where there are almost always people online that can help you in the Slack channel. -If you need help with using Netbox or developing for it or against it's API you may find the `#netbox` channel on the same Slack instance very helpful. +If you need help with using NetBox or developing for it or against it's API you may find the `#netbox` channel on the same Slack instance very helpful. [netbox-community]: https://github.com/netbox-community/netbox-docker/discussions @@ -102,9 +102,9 @@ This project relies only on *Docker* and *docker-compose* meeting these requirem To check the version installed on your system run `docker --version` and `docker-compose --version`. -## Use a Specific Netbox Version +## Use a Specific NetBox Version -The `docker-compose.yml` file is prepared to run a specific version of Netbox, instead of `latest`. +The `docker-compose.yml` file is prepared to run a specific version of NetBox, instead of `latest`. To use this feature, set and export the environment-variable `VERSION` before launching `docker-compose`, as shown below. `VERSION` may be set to the name of [any tag of the `netboxcommunity/netbox` Docker image on Docker Hub][netbox-dockerhub] or [Quay.io][netbox-quayio]. @@ -115,7 +115,7 @@ docker-compose pull netbox docker-compose up -d ``` -You can also build a specific version of the Netbox Docker image yourself. +You can also build a specific version of the NetBox Docker image yourself. `VERSION` can be any valid [git ref][git-ref] in that case. ```bash @@ -151,7 +151,7 @@ For more details on custom builds [consult our wiki][netbox-docker-wiki-build]. ## Tests We have a test script. -It runs Netbox's own unit tests and ensures that all initializers work: +It runs NetBox's own unit tests and ensures that all initializers work: ```bash IMAGE=netboxcommunity/netbox:latest ./test.sh diff --git a/build-next.sh b/build-next.sh index f134333..88f290b 100755 --- a/build-next.sh +++ b/build-next.sh @@ -1,5 +1,5 @@ #!/bin/bash -# Builds develop, develop-* and master branches of Netbox +# Builds develop, develop-* and master branches of NetBox echo "▶️ $0 $*" diff --git a/build.sh b/build.sh index 897f74e..408ad0c 100755 --- a/build.sh +++ b/build.sh @@ -1,5 +1,5 @@ #!/bin/bash -# Clones the Netbox repository with git from Github and builds the Dockerfile +# Clones the NetBox repository with git from Github and builds the Dockerfile echo "▶️ $0 $*" @@ -106,7 +106,7 @@ else fi ### -# Variables for fetching the Netbox source +# Variables for fetching the NetBox source ### SRC_ORG="${SRC_ORG-netbox-community}" SRC_REPO="${SRC_REPO-netbox}" @@ -115,10 +115,10 @@ URL="${URL-https://github.com/${SRC_ORG}/${SRC_REPO}.git}" NETBOX_PATH="${NETBOX_PATH-.netbox}" ### -# Fetching the Netbox source +# Fetching the NetBox source ### if [ "${2}" != "--push-only" ] && [ -z "${SKIP_GIT}" ] ; then - echo "🌐 Checking out '${NETBOX_BRANCH}' of Netbox from the url '${URL}' into '${NETBOX_PATH}'" + echo "🌐 Checking out '${NETBOX_BRANCH}' of NetBox from the url '${URL}' into '${NETBOX_PATH}'" if [ ! -d "${NETBOX_PATH}" ]; then $DRY git clone -q --depth 10 -b "${NETBOX_BRANCH}" "${URL}" "${NETBOX_PATH}" fi @@ -135,7 +135,7 @@ if [ "${2}" != "--push-only" ] && [ -z "${SKIP_GIT}" ] ; then $DRY git checkout -qf FETCH_HEAD $DRY git prune ) - echo "✅ Checked out Netbox" + echo "✅ Checked out NetBox" fi ### diff --git a/docker/docker-entrypoint.sh b/docker/docker-entrypoint.sh index 0765a4a..6d522bf 100755 --- a/docker/docker-entrypoint.sh +++ b/docker/docker-entrypoint.sh @@ -1,10 +1,10 @@ #!/bin/bash -# Runs on every start of the Netbox Docker container +# Runs on every start of the NetBox Docker container # Stop when an error occures set -e -# Allows Netbox to be run as non-root users +# Allows NetBox to be run as non-root users umask 002 # Load correct Python3 env diff --git a/test.sh b/test.sh index cc033b0..2013690 100755 --- a/test.sh +++ b/test.sh @@ -1,5 +1,5 @@ #!/bin/bash -# Runs the original Netbox unit tests and tests whether all initializers work. +# Runs the original NetBox unit tests and tests whether all initializers work. # Usage: # ./test.sh latest # ./test.sh v2.9.7 @@ -55,7 +55,7 @@ test_setup() { } test_netbox_unit_tests() { - echo "⏱ Running Netbox Unit Tests" + echo "⏱ Running NetBox Unit Tests" SKIP_STARTUP_SCRIPTS=true $doco run --rm netbox ./manage.py test } From 5343eaae6545443c2583a2060c30b3a1dedd2af5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20M=C3=A4der?= Date: Mon, 8 Feb 2021 09:02:14 +0100 Subject: [PATCH 04/28] Fix building cryptography pip --- Dockerfile | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/Dockerfile b/Dockerfile index 3d5d5ac..e8261fb 100644 --- a/Dockerfile +++ b/Dockerfile @@ -4,23 +4,31 @@ FROM ${FROM} as builder RUN apk add --no-cache \ bash \ build-base \ + cargo \ ca-certificates \ cyrus-sasl-dev \ graphviz \ jpeg-dev \ libevent-dev \ libffi-dev \ + libressl-dev \ libxslt-dev \ + musl-dev \ openldap-dev \ postgresql-dev \ py3-pip \ python3-dev \ - && python3 -m venv /opt/netbox/venv \ - && /opt/netbox/venv/bin/python3 -m pip install --upgrade pip setuptools + && python3 -m venv /opt/netbox/venv \ + && /opt/netbox/venv/bin/python3 -m pip install --upgrade \ + pip \ + setuptools \ + wheel ARG NETBOX_PATH COPY ${NETBOX_PATH}/requirements.txt requirements-container.txt / -RUN /opt/netbox/venv/bin/pip install -r /requirements.txt -r /requirements-container.txt +RUN /opt/netbox/venv/bin/pip install \ + -r /requirements.txt \ + -r /requirements-container.txt ### # Main stage From 187ae4b2a1a4cc524bd87c95894a7d18966d3d15 Mon Sep 17 00:00:00 2001 From: Schylar Utleye Date: Fri, 18 Dec 2020 23:22:41 -0600 Subject: [PATCH 05/28] add custom link initializers --- initializers/custom_links.yml | 21 +++++++++++++++++++++ startup_scripts/280_custom_links.py | 24 ++++++++++++++++++++++++ 2 files changed, 45 insertions(+) create mode 100644 initializers/custom_links.yml create mode 100644 startup_scripts/280_custom_links.py diff --git a/initializers/custom_links.yml b/initializers/custom_links.yml new file mode 100644 index 0000000..a18acb5 --- /dev/null +++ b/initializers/custom_links.yml @@ -0,0 +1,21 @@ +## Possible Choices: +## new_window: +## - True +## - False +## content_type_id: +## - device +## - site +## - any-other-content-type +## +## Examples: + +# - name: link_to_repo + # text: 'Link to docker repository' + # url: 'https://github.com/netbox-community/netbox-docker' + # new_window: False + # content_type: device +# - name: link_to_localhost + # text: 'Link to the users localhost' + # url: 'http://localhost' + # new_window: True + # content_type: device diff --git a/startup_scripts/280_custom_links.py b/startup_scripts/280_custom_links.py new file mode 100644 index 0000000..6e8bb04 --- /dev/null +++ b/startup_scripts/280_custom_links.py @@ -0,0 +1,24 @@ +from django.contrib.contenttypes.models import ContentType +from extras.models import CustomLink +from startup_script_utils import load_yaml +import sys + + +custom_links = load_yaml('/opt/netbox/initializers/custom_links.yml') + +if custom_links is None: + sys.exit() + +def get_content_type_id(content_type_str): + for type in ContentType.objects.all(): + if type.name == content_type_str: + return type.id + +for link in custom_links: + content_type = link.pop('content_type') + link['content_type_id'] = get_content_type_id(content_type) + if link['content_type_id'] is None: + print("⚠️ Error determining content type id for user declared var: {0}".format(content_type)) + else: + CustomLink(**link).save() + From 7112a88359dac7951d03fb3beb73bcbd65661f23 Mon Sep 17 00:00:00 2001 From: Schylar Utleye Date: Sun, 20 Dec 2020 21:50:07 +0000 Subject: [PATCH 06/28] add webhook initializer --- initializers/webhooks.yml | 29 +++++++++++++++++++++++++++++ startup_scripts/290_webhooks.py | 28 ++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+) create mode 100644 initializers/webhooks.yml create mode 100644 startup_scripts/290_webhooks.py diff --git a/initializers/webhooks.yml b/initializers/webhooks.yml new file mode 100644 index 0000000..ef4fa5d --- /dev/null +++ b/initializers/webhooks.yml @@ -0,0 +1,29 @@ +## Possible Choices: +## object_types: +## - device +## - site +## - any-other-content-type +## types: +## - type_create +## - type_update +## - type_delete +## Examples: + +# - name: device_creation + # payload_url: 'https://github.com/netbox-community/netbox-docker' + # object_types: + # - device + # - cable + # type_create: True +# - name: device_update + # payload_url: 'https://google.com' + # object_types: + # - device + # type_update: True +- name: device_dele1te + payload_url: 'https://gitlab.com' + object_types: + - device + type_delete: True + + diff --git a/startup_scripts/290_webhooks.py b/startup_scripts/290_webhooks.py new file mode 100644 index 0000000..73b953b --- /dev/null +++ b/startup_scripts/290_webhooks.py @@ -0,0 +1,28 @@ +from django.contrib.contenttypes.models import ContentType +from extras.models import Webhook +from startup_script_utils import load_yaml +import sys + + +webhooks = load_yaml('/opt/netbox/initializers/webhooks.yml') + +if webhooks is None: + sys.exit() + +def get_content_type_id(content_type_str): + for type in ContentType.objects.all(): + if type.name == content_type_str: + return type.id + +for hook in webhooks: + obj_types = hook.pop('object_types') + obj_type_ids = [] + for obj in obj_types: + obj_type_ids.append(get_content_type_id(obj)) + if obj_type_ids is None: + print("⚠️ Error determining content type id for user declared var: {0}".format(obj_type)) + else: + webhook = Webhook(**hook) + webhook.save() + webhook.obj_type.set(obj_type_ids) + # webhook.save() \ No newline at end of file From d0c786e83149448af7221f243101863046fcb5e0 Mon Sep 17 00:00:00 2001 From: Schylar Utleye Date: Sun, 20 Dec 2020 15:53:58 -0600 Subject: [PATCH 07/28] Update webhooks.yml fix comment --- initializers/webhooks.yml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/initializers/webhooks.yml b/initializers/webhooks.yml index ef4fa5d..ba5b777 100644 --- a/initializers/webhooks.yml +++ b/initializers/webhooks.yml @@ -20,10 +20,10 @@ # object_types: # - device # type_update: True -- name: device_dele1te - payload_url: 'https://gitlab.com' - object_types: - - device - type_delete: True +#- name: device_delete +# payload_url: 'https://gitlab.com' +# object_types: +# - device +# type_delete: True From a3cf645dc5b759cc7b32b441a28c3e82515effa0 Mon Sep 17 00:00:00 2001 From: Schylar Utleye Date: Fri, 15 Jan 2021 22:12:57 -0600 Subject: [PATCH 08/28] fix webhook initializer yaml and webhook startup script to work with latest netbox release --- initializers/webhooks.yml | 28 ++++++++++++++-------------- startup_scripts/290_webhooks.py | 13 +++++++------ 2 files changed, 21 insertions(+), 20 deletions(-) diff --git a/initializers/webhooks.yml b/initializers/webhooks.yml index ba5b777..9d78907 100644 --- a/initializers/webhooks.yml +++ b/initializers/webhooks.yml @@ -10,20 +10,20 @@ ## Examples: # - name: device_creation - # payload_url: 'https://github.com/netbox-community/netbox-docker' - # object_types: - # - device - # - cable - # type_create: True +# payload_url: 'https://github.com/netbox-community/netbox-docker' +# object_types: +# - device +# - cable +# type_create: True # - name: device_update - # payload_url: 'https://google.com' - # object_types: - # - device - # type_update: True -#- name: device_delete -# payload_url: 'https://gitlab.com' -# object_types: -# - device -# type_delete: True +# payload_url: 'https://google.com' +# object_types: +# - device +# type_update: True +# - name: device_delete +# payload_url: 'https://gitlab.com1' +# object_types: +# - device +# type_delete: True diff --git a/startup_scripts/290_webhooks.py b/startup_scripts/290_webhooks.py index 73b953b..c4d7299 100644 --- a/startup_scripts/290_webhooks.py +++ b/startup_scripts/290_webhooks.py @@ -10,9 +10,7 @@ if webhooks is None: sys.exit() def get_content_type_id(content_type_str): - for type in ContentType.objects.all(): - if type.name == content_type_str: - return type.id + return ContentType.objects.get(model=content_type_str).id for hook in webhooks: obj_types = hook.pop('object_types') @@ -23,6 +21,9 @@ for hook in webhooks: print("⚠️ Error determining content type id for user declared var: {0}".format(obj_type)) else: webhook = Webhook(**hook) - webhook.save() - webhook.obj_type.set(obj_type_ids) - # webhook.save() \ No newline at end of file + if not Webhook.objects.filter(name=webhook.name): + webhook.save() + webhook.content_types.set(obj_type_ids) + print(" Created Webhook {0}".format(webhook.name)) + else: + print(" Skipping Webhook {0}, already exists".format(webhook.name)) From 618feff63a54bc3e324f6e97a6dbffd95974241e Mon Sep 17 00:00:00 2001 From: Schylar Utleye Date: Fri, 15 Jan 2021 23:12:03 -0600 Subject: [PATCH 09/28] add error handling for webhook and custom links. fix initializer comments --- initializers/custom_links.yml | 20 ++++++++++---------- startup_scripts/280_custom_links.py | 19 ++++++++++++------- startup_scripts/290_webhooks.py | 14 ++++++++------ 3 files changed, 30 insertions(+), 23 deletions(-) diff --git a/initializers/custom_links.yml b/initializers/custom_links.yml index a18acb5..4f3ac01 100644 --- a/initializers/custom_links.yml +++ b/initializers/custom_links.yml @@ -9,13 +9,13 @@ ## ## Examples: -# - name: link_to_repo - # text: 'Link to docker repository' - # url: 'https://github.com/netbox-community/netbox-docker' - # new_window: False - # content_type: device -# - name: link_to_localhost - # text: 'Link to the users localhost' - # url: 'http://localhost' - # new_window: True - # content_type: device +# - name: link_to_repo +# text: 'Link to docker repository' +# url: 'https://github.com/netbox-community/netbox-docker' +# new_window: False +# content_type: device +# - name: link_to_localhost +# text: 'Link to the users localhost' +# url: 'http://localhost' +# new_window: True +# content_type: device diff --git a/startup_scripts/280_custom_links.py b/startup_scripts/280_custom_links.py index 6e8bb04..dc75839 100644 --- a/startup_scripts/280_custom_links.py +++ b/startup_scripts/280_custom_links.py @@ -10,15 +10,20 @@ if custom_links is None: sys.exit() def get_content_type_id(content_type_str): - for type in ContentType.objects.all(): - if type.name == content_type_str: - return type.id + try: + id = ContentType.objects.get(model=content_type_str).id + return id + except ContentType.DoesNotExist: + print(" Error determining content type id for user declared var: {0}".format(content_type_str)) for link in custom_links: content_type = link.pop('content_type') link['content_type_id'] = get_content_type_id(content_type) - if link['content_type_id'] is None: - print("⚠️ Error determining content type id for user declared var: {0}".format(content_type)) - else: - CustomLink(**link).save() + if link['content_type_id'] is not None: + custom_link = CustomLink(**link) + if not CustomLink.objects.filter(name=custom_link.name): + custom_link.save() + print(" Created Custom Link {0}".format(custom_link.name)) + else: + print("⚠️ Skipping Custom Link {0}, already exists".format(custom_link.name)) diff --git a/startup_scripts/290_webhooks.py b/startup_scripts/290_webhooks.py index c4d7299..3b40fcf 100644 --- a/startup_scripts/290_webhooks.py +++ b/startup_scripts/290_webhooks.py @@ -10,20 +10,22 @@ if webhooks is None: sys.exit() def get_content_type_id(content_type_str): - return ContentType.objects.get(model=content_type_str).id + try: + id = ContentType.objects.get(model=content_type_str).id + return id + except ContentType.DoesNotExist: + print("⚠️ Error determining content type id for user declared var: {0}".format(content_type_str)) for hook in webhooks: obj_types = hook.pop('object_types') obj_type_ids = [] for obj in obj_types: obj_type_ids.append(get_content_type_id(obj)) - if obj_type_ids is None: - print("⚠️ Error determining content type id for user declared var: {0}".format(obj_type)) - else: + if obj_type_ids is not None: webhook = Webhook(**hook) if not Webhook.objects.filter(name=webhook.name): webhook.save() webhook.content_types.set(obj_type_ids) - print(" Created Webhook {0}".format(webhook.name)) + print("🖥️ Created Webhook {0}".format(webhook.name)) else: - print(" Skipping Webhook {0}, already exists".format(webhook.name)) + print("⚠️ Skipping Webhook {0}, already exists".format(webhook.name)) From 8321449cc084f9ebf2caf1578206d16b189b9532 Mon Sep 17 00:00:00 2001 From: Schylar Utleye Date: Fri, 15 Jan 2021 23:34:44 -0600 Subject: [PATCH 10/28] add icons to help messages --- startup_scripts/280_custom_links.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/startup_scripts/280_custom_links.py b/startup_scripts/280_custom_links.py index dc75839..b014a29 100644 --- a/startup_scripts/280_custom_links.py +++ b/startup_scripts/280_custom_links.py @@ -14,7 +14,7 @@ def get_content_type_id(content_type_str): id = ContentType.objects.get(model=content_type_str).id return id except ContentType.DoesNotExist: - print(" Error determining content type id for user declared var: {0}".format(content_type_str)) + print("⚠️ Error determining content type id for user declared var: {0}".format(content_type_str)) for link in custom_links: content_type = link.pop('content_type') @@ -23,7 +23,7 @@ for link in custom_links: custom_link = CustomLink(**link) if not CustomLink.objects.filter(name=custom_link.name): custom_link.save() - print(" Created Custom Link {0}".format(custom_link.name)) + print("🖥️ Created Custom Link {0}".format(custom_link.name)) else: print("⚠️ Skipping Custom Link {0}, already exists".format(custom_link.name)) From fe811f37bd03f52222ff273fb7620989155e7a45 Mon Sep 17 00:00:00 2001 From: Schylar Utley Date: Sat, 16 Jan 2021 00:06:04 -0600 Subject: [PATCH 11/28] replace loop with list comprehension --- startup_scripts/290_webhooks.py | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/startup_scripts/290_webhooks.py b/startup_scripts/290_webhooks.py index 3b40fcf..315cf44 100644 --- a/startup_scripts/290_webhooks.py +++ b/startup_scripts/290_webhooks.py @@ -18,9 +18,7 @@ def get_content_type_id(content_type_str): for hook in webhooks: obj_types = hook.pop('object_types') - obj_type_ids = [] - for obj in obj_types: - obj_type_ids.append(get_content_type_id(obj)) + obj_type_ids = [ get_content_type_id(obj) for obj in obj_types ] if obj_type_ids is not None: webhook = Webhook(**hook) if not Webhook.objects.filter(name=webhook.name): From 07a0b1d7ef38f472fef745a241e6e425d2a3de02 Mon Sep 17 00:00:00 2001 From: Schylar Utley Date: Wed, 20 Jan 2021 15:47:34 -0600 Subject: [PATCH 12/28] Update startup_scripts/280_custom_links.py MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Christian Mäder --- startup_scripts/280_custom_links.py | 3 --- 1 file changed, 3 deletions(-) diff --git a/startup_scripts/280_custom_links.py b/startup_scripts/280_custom_links.py index b014a29..7545a0b 100644 --- a/startup_scripts/280_custom_links.py +++ b/startup_scripts/280_custom_links.py @@ -24,6 +24,3 @@ for link in custom_links: if not CustomLink.objects.filter(name=custom_link.name): custom_link.save() print("🖥️ Created Custom Link {0}".format(custom_link.name)) - else: - print("⚠️ Skipping Custom Link {0}, already exists".format(custom_link.name)) - From 95f4d7856a12f8b7ec795784eef2c43e5c3fcbfe Mon Sep 17 00:00:00 2001 From: Schylar Utley Date: Wed, 20 Jan 2021 15:47:44 -0600 Subject: [PATCH 13/28] Update startup_scripts/280_custom_links.py MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Christian Mäder --- startup_scripts/280_custom_links.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/startup_scripts/280_custom_links.py b/startup_scripts/280_custom_links.py index 7545a0b..83da718 100644 --- a/startup_scripts/280_custom_links.py +++ b/startup_scripts/280_custom_links.py @@ -11,8 +11,7 @@ if custom_links is None: def get_content_type_id(content_type_str): try: - id = ContentType.objects.get(model=content_type_str).id - return id + return ContentType.objects.get(model=content_type_str).id except ContentType.DoesNotExist: print("⚠️ Error determining content type id for user declared var: {0}".format(content_type_str)) From f88f4e15790f2753d0f043f6ddda7b8e72bef46b Mon Sep 17 00:00:00 2001 From: Schylar Utley Date: Wed, 20 Jan 2021 15:47:53 -0600 Subject: [PATCH 14/28] Update startup_scripts/290_webhooks.py MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Christian Mäder --- startup_scripts/290_webhooks.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/startup_scripts/290_webhooks.py b/startup_scripts/290_webhooks.py index 315cf44..8fc924e 100644 --- a/startup_scripts/290_webhooks.py +++ b/startup_scripts/290_webhooks.py @@ -25,5 +25,3 @@ for hook in webhooks: webhook.save() webhook.content_types.set(obj_type_ids) print("🖥️ Created Webhook {0}".format(webhook.name)) - else: - print("⚠️ Skipping Webhook {0}, already exists".format(webhook.name)) From 52c51b5f99e22e73ad98d0cb5d17b72bb98b0db9 Mon Sep 17 00:00:00 2001 From: Schylar Utley Date: Wed, 20 Jan 2021 15:48:17 -0600 Subject: [PATCH 15/28] Update initializers/custom_links.yml MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Christian Mäder --- initializers/custom_links.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/initializers/custom_links.yml b/initializers/custom_links.yml index 4f3ac01..1e621ad 100644 --- a/initializers/custom_links.yml +++ b/initializers/custom_links.yml @@ -2,7 +2,7 @@ ## new_window: ## - True ## - False -## content_type_id: +## content_type: ## - device ## - site ## - any-other-content-type From 744f0e57ad5a47ed0e216158e5290b1bf59cdffe Mon Sep 17 00:00:00 2001 From: Schylar Utley Date: Wed, 20 Jan 2021 15:48:48 -0600 Subject: [PATCH 16/28] Update initializers/webhooks.yml MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Christian Mäder --- initializers/webhooks.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/initializers/webhooks.yml b/initializers/webhooks.yml index 9d78907..ff214a5 100644 --- a/initializers/webhooks.yml +++ b/initializers/webhooks.yml @@ -25,5 +25,3 @@ # object_types: # - device # type_delete: True - - From 2e5d84612d1ee78e7f05917cdc95028fef89c1c3 Mon Sep 17 00:00:00 2001 From: Schylar Utley Date: Wed, 20 Jan 2021 21:25:52 -0600 Subject: [PATCH 17/28] Update initializers/custom_links.yml MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Christian Mäder --- initializers/custom_links.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/initializers/custom_links.yml b/initializers/custom_links.yml index 1e621ad..334efc6 100644 --- a/initializers/custom_links.yml +++ b/initializers/custom_links.yml @@ -10,10 +10,10 @@ ## Examples: # - name: link_to_repo -# text: 'Link to docker repository' -# url: 'https://github.com/netbox-community/netbox-docker' -# new_window: False -# content_type: device +# text: 'Link to docker repository' +# url: 'https://github.com/netbox-community/netbox-docker' +# new_window: False +# content_type: device # - name: link_to_localhost # text: 'Link to the users localhost' # url: 'http://localhost' From e4e2c788a98a01277c64e5d1d30677a3839c3985 Mon Sep 17 00:00:00 2001 From: Schylar Utley Date: Wed, 20 Jan 2021 21:42:24 -0600 Subject: [PATCH 18/28] Update 290_webhooks.py Move to a more standard method of object handling --- startup_scripts/290_webhooks.py | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/startup_scripts/290_webhooks.py b/startup_scripts/290_webhooks.py index 8fc924e..f6d480b 100644 --- a/startup_scripts/290_webhooks.py +++ b/startup_scripts/290_webhooks.py @@ -20,8 +20,7 @@ for hook in webhooks: obj_types = hook.pop('object_types') obj_type_ids = [ get_content_type_id(obj) for obj in obj_types ] if obj_type_ids is not None: - webhook = Webhook(**hook) - if not Webhook.objects.filter(name=webhook.name): - webhook.save() + webhook, created = Webhook.objects.get_or_create(**hook) + if created: webhook.content_types.set(obj_type_ids) print("🖥️ Created Webhook {0}".format(webhook.name)) From 16ae0633210f4692cff2eeb1ccd20230bb93d3d7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20M=C3=A4der?= Date: Mon, 8 Feb 2021 10:35:31 +0100 Subject: [PATCH 19/28] Adjust to repository standards --- initializers/custom_links.yml | 20 +++++----- initializers/webhooks.yml | 54 +++++++++++++------------- startup_scripts/280_custom_links.py | 20 ++++++---- startup_scripts/290_webhooks.py | 60 ++++++++++++++++------------- 4 files changed, 83 insertions(+), 71 deletions(-) diff --git a/initializers/custom_links.yml b/initializers/custom_links.yml index 334efc6..f1b275c 100644 --- a/initializers/custom_links.yml +++ b/initializers/custom_links.yml @@ -9,13 +9,13 @@ ## ## Examples: -# - name: link_to_repo -# text: 'Link to docker repository' -# url: 'https://github.com/netbox-community/netbox-docker' -# new_window: False -# content_type: device -# - name: link_to_localhost -# text: 'Link to the users localhost' -# url: 'http://localhost' -# new_window: True -# content_type: device +# - name: link_to_repo +# text: 'Link to Netbox Docker' +# url: 'https://github.com/netbox-community/netbox-docker' +# new_window: False +# content_type: device +# - name: link_to_localhost +# text: 'Link to localhost' +# url: 'http://localhost' +# new_window: True +# content_type: device diff --git a/initializers/webhooks.yml b/initializers/webhooks.yml index ff214a5..deb1b39 100644 --- a/initializers/webhooks.yml +++ b/initializers/webhooks.yml @@ -1,27 +1,27 @@ -## Possible Choices: -## object_types: -## - device -## - site -## - any-other-content-type -## types: -## - type_create -## - type_update -## - type_delete -## Examples: - -# - name: device_creation -# payload_url: 'https://github.com/netbox-community/netbox-docker' -# object_types: -# - device -# - cable -# type_create: True -# - name: device_update -# payload_url: 'https://google.com' -# object_types: -# - device -# type_update: True -# - name: device_delete -# payload_url: 'https://gitlab.com1' -# object_types: -# - device -# type_delete: True +## Possible Choices: +## object_types: +## - device +## - site +## - any-other-content-type +## types: +## - type_create +## - type_update +## - type_delete +## Examples: + +# - name: device_creation +# payload_url: 'http://localhost:8080' +# object_types: +# - device +# - cable +# type_create: True +# - name: device_update +# payload_url: 'http://localhost:8080' +# object_types: +# - device +# type_update: True +# - name: device_delete +# payload_url: 'http://localhost:8080' +# object_types: +# - device +# type_delete: True diff --git a/startup_scripts/280_custom_links.py b/startup_scripts/280_custom_links.py index 83da718..b2a7df5 100644 --- a/startup_scripts/280_custom_links.py +++ b/startup_scripts/280_custom_links.py @@ -9,17 +9,21 @@ custom_links = load_yaml('/opt/netbox/initializers/custom_links.yml') if custom_links is None: sys.exit() -def get_content_type_id(content_type_str): + +def get_content_type_id(content_type): try: - return ContentType.objects.get(model=content_type_str).id + return ContentType.objects.get(model=content_type).id except ContentType.DoesNotExist: - print("⚠️ Error determining content type id for user declared var: {0}".format(content_type_str)) + pass + for link in custom_links: content_type = link.pop('content_type') link['content_type_id'] = get_content_type_id(content_type) - if link['content_type_id'] is not None: - custom_link = CustomLink(**link) - if not CustomLink.objects.filter(name=custom_link.name): - custom_link.save() - print("🖥️ Created Custom Link {0}".format(custom_link.name)) + if link['content_type_id'] is None: + print("⚠️ Unable to create Custom Link '{0}': The content_type '{1}' is unknown".format(link.name, content_type)) + continue + + custom_link, created = CustomLink.objects.get_or_create(**link) + if created: + print("🔗 Created Custom Link '{0}'".format(custom_link.name)) diff --git a/startup_scripts/290_webhooks.py b/startup_scripts/290_webhooks.py index f6d480b..ea8352e 100644 --- a/startup_scripts/290_webhooks.py +++ b/startup_scripts/290_webhooks.py @@ -1,26 +1,34 @@ -from django.contrib.contenttypes.models import ContentType -from extras.models import Webhook -from startup_script_utils import load_yaml -import sys - - -webhooks = load_yaml('/opt/netbox/initializers/webhooks.yml') - -if webhooks is None: - sys.exit() - -def get_content_type_id(content_type_str): - try: - id = ContentType.objects.get(model=content_type_str).id - return id - except ContentType.DoesNotExist: - print("⚠️ Error determining content type id for user declared var: {0}".format(content_type_str)) - -for hook in webhooks: - obj_types = hook.pop('object_types') - obj_type_ids = [ get_content_type_id(obj) for obj in obj_types ] - if obj_type_ids is not None: - webhook, created = Webhook.objects.get_or_create(**hook) - if created: - webhook.content_types.set(obj_type_ids) - print("🖥️ Created Webhook {0}".format(webhook.name)) +from django.contrib.contenttypes.models import ContentType +from extras.models import Webhook +from startup_script_utils import load_yaml +import sys + + +webhooks = load_yaml('/opt/netbox/initializers/webhooks.yml') + +if webhooks is None: + sys.exit() + + +def get_content_type_id(hook_name, content_type): + try: + return ContentType.objects.get(model=content_type).id + except ContentType.DoesNotExist as ex: + print("⚠️ Webhook '{0}': The object_type '{1}' is unknown.".format(hook_name, content_type)) + raise ex + + +for hook in webhooks: + obj_types = hook.pop('object_types') + + try: + obj_type_ids = [get_content_type_id(hook['name'], obj) for obj in obj_types] + except ContentType.DoesNotExist: + continue + + webhook, created = Webhook.objects.get_or_create(**hook) + if created: + webhook.content_types.set(obj_type_ids) + webhook.save() + + print("🪝 Created Webhook {0}".format(webhook.name)) From 0d25aff744a44e93693c20629a06245a72ddd380 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20M=C3=A4der?= Date: Mon, 8 Feb 2021 10:54:08 +0100 Subject: [PATCH 20/28] Add editorconfig --- .editorconfig | 11 +++++++ docker/nginx-unit.json | 68 +++++++++++++++++++++--------------------- 2 files changed, 45 insertions(+), 34 deletions(-) create mode 100644 .editorconfig diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 0000000..dba40a8 --- /dev/null +++ b/.editorconfig @@ -0,0 +1,11 @@ +root = true + +[*] +end_of_line = lf +insert_final_newline = true +charset = utf-8 +indent_style = space +indent_size = 2 + +[configuration/**.py] +indent_size = 4 diff --git a/docker/nginx-unit.json b/docker/nginx-unit.json index 80673ae..fbe8c2b 100644 --- a/docker/nginx-unit.json +++ b/docker/nginx-unit.json @@ -1,40 +1,40 @@ { - "listeners": { - "*:8080": { - "pass": "routes" - } + "listeners": { + "*:8080": { + "pass": "routes" + } + }, + + "routes": [ + { + "match": { + "uri": "/static/*" + }, + "action": { + "share": "/opt/netbox/netbox" + } }, - "routes": [ - { - "match": { - "uri": "/static/*" - }, - "action": { - "share": "/opt/netbox/netbox" - } - }, + { + "action": { + "pass": "applications/netbox" + } + } + ], - { - "action": { - "pass": "applications/netbox" - } - } - ], + "applications": { + "netbox": { + "type": "python 3", + "path": "/opt/netbox/netbox/", + "module": "netbox.wsgi", + "home": "/opt/netbox/venv", + "processes": { + "max": 4, + "spare": 1, + "idle_timeout": 120 + } + } + }, - "applications": { - "netbox": { - "type": "python 3", - "path": "/opt/netbox/netbox/", - "module": "netbox.wsgi", - "home": "/opt/netbox/venv", - "processes": { - "max": 4, - "spare": 1, - "idle_timeout": 120 - } - } - }, - - "access_log": "/dev/stdout" + "access_log": "/dev/stdout" } From 493fc60401fe7fb7ac2c07a370838b796cc269bb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20M=C3=A4der?= Date: Mon, 8 Feb 2021 11:59:33 +0100 Subject: [PATCH 21/28] Use black as formatter for python files --- .editorconfig | 2 +- Dockerfile | 4 +- configuration/configuration.py | 17 ++-- configuration/ldap/ldap_config.py | 15 ++-- docker/configuration.docker.py | 81 ++++++++--------- docker/ldap_config.docker.py | 28 +++--- initializers/custom_fields.yml | 28 +++--- pyproject.toml | 21 +++++ startup_scripts/000_users.py | 23 ++--- startup_scripts/010_groups.py | 22 ++--- startup_scripts/020_custom_fields.py | 76 ++++++++-------- startup_scripts/020_tags.py | 28 +++--- startup_scripts/030_regions.py | 29 +++--- startup_scripts/040_sites.py | 31 +++---- startup_scripts/050_manufacturers.py | 15 ++-- startup_scripts/060_device_types.py | 43 ++++----- startup_scripts/070_rack_roles.py | 28 +++--- startup_scripts/075_rack_groups.py | 28 +++--- startup_scripts/080_racks.py | 46 +++++----- startup_scripts/090_device_roles.py | 28 +++--- startup_scripts/100_platforms.py | 27 +++--- startup_scripts/110_tenant_groups.py | 15 ++-- startup_scripts/120_tenants.py | 30 +++---- startup_scripts/130_cluster_types.py | 15 ++-- startup_scripts/135_cluster_groups.py | 15 ++-- startup_scripts/135_clusters.py | 46 +++++----- startup_scripts/140_clusters.py | 46 +++++----- startup_scripts/140_devices.py | 56 ++++++------ startup_scripts/145_devices.py | 56 ++++++------ startup_scripts/150_rirs.py | 15 ++-- startup_scripts/160_aggregates.py | 44 +++++----- startup_scripts/165_cluster_groups.py | 15 ++-- startup_scripts/175_route_targets.py | 30 +++---- startup_scripts/180_vrfs.py | 30 +++---- startup_scripts/190_prefix_vlan_roles.py | 15 ++-- startup_scripts/200_vlan_groups.py | 30 +++---- startup_scripts/210_vlans.py | 38 ++++---- startup_scripts/220_prefixes.py | 42 ++++----- startup_scripts/230_virtual_machines.py | 52 ++++++----- .../240_virtualization_interfaces.py | 28 +++--- startup_scripts/250_dcim_interfaces.py | 30 +++---- startup_scripts/260_ip_addresses.py | 88 ++++++++++--------- startup_scripts/270_primary_ips.py | 61 +++++++------ startup_scripts/280_custom_links.py | 36 ++++---- startup_scripts/280_providers.py | 19 ++-- startup_scripts/290_circuit_types.py | 19 ++-- startup_scripts/290_webhooks.py | 38 ++++---- startup_scripts/300_circuits.py | 48 +++++----- startup_scripts/310_secret_roles.py | 15 ++-- startup_scripts/320_services.py | 33 +++---- startup_scripts/330_power_panels.py | 45 +++++----- startup_scripts/340_power_feeds.py | 45 +++++----- startup_scripts/__main__.py | 34 +++---- .../startup_script_utils/__init__.py | 2 +- .../startup_script_utils/custom_fields.py | 21 ++--- .../startup_script_utils/load_yaml.py | 13 +-- .../startup_script_utils/permissions.py | 30 ++++--- 57 files changed, 915 insertions(+), 900 deletions(-) create mode 100644 pyproject.toml diff --git a/.editorconfig b/.editorconfig index dba40a8..7f9f55d 100644 --- a/.editorconfig +++ b/.editorconfig @@ -7,5 +7,5 @@ charset = utf-8 indent_style = space indent_size = 2 -[configuration/**.py] +[*.py] indent_size = 4 diff --git a/Dockerfile b/Dockerfile index e8261fb..280a9c6 100644 --- a/Dockerfile +++ b/Dockerfile @@ -18,8 +18,8 @@ RUN apk add --no-cache \ postgresql-dev \ py3-pip \ python3-dev \ - && python3 -m venv /opt/netbox/venv \ - && /opt/netbox/venv/bin/python3 -m pip install --upgrade \ + && python3 -m venv /opt/netbox/venv \ + && /opt/netbox/venv/bin/python3 -m pip install --upgrade \ pip \ setuptools \ wheel diff --git a/configuration/configuration.py b/configuration/configuration.py index 36517f0..727f7b3 100644 --- a/configuration/configuration.py +++ b/configuration/configuration.py @@ -5,9 +5,8 @@ #### import re - -from os.path import dirname, abspath, join from os import environ +from os.path import abspath, dirname, join # For reference see https://netbox.readthedocs.io/en/stable/configuration/ # Based on https://github.com/netbox-community/netbox/blob/master/netbox/netbox/configuration.example.py @@ -39,16 +38,16 @@ ALLOWED_HOSTS = environ.get('ALLOWED_HOSTS', '*').split(' ') # PostgreSQL database configuration. See the Django documentation for a complete list of available parameters: # https://docs.djangoproject.com/en/stable/ref/settings/#databases DATABASE = { - 'NAME': environ.get('DB_NAME', 'netbox'), # Database name - 'USER': environ.get('DB_USER', ''), # PostgreSQL username + 'NAME': environ.get('DB_NAME', 'netbox'), # Database name + 'USER': environ.get('DB_USER', ''), # PostgreSQL username 'PASSWORD': _read_secret('db_password', environ.get('DB_PASSWORD', '')), - # PostgreSQL password - 'HOST': environ.get('DB_HOST', 'localhost'), # Database server - 'PORT': environ.get('DB_PORT', ''), # Database port (leave blank for default) + # PostgreSQL password + 'HOST': environ.get('DB_HOST', 'localhost'), # Database server + 'PORT': environ.get('DB_PORT', ''), # Database port (leave blank for default) 'OPTIONS': {'sslmode': environ.get('DB_SSLMODE', 'prefer')}, - # Database connection SSLMODE + # Database connection SSLMODE 'CONN_MAX_AGE': int(environ.get('DB_CONN_MAX_AGE', '300')), - # Max database connection age + # Max database connection age } # Redis database settings. Redis is used for caching and for queuing background tasks such as webhook events. A separate diff --git a/configuration/ldap/ldap_config.py b/configuration/ldap/ldap_config.py index 1ed599f..3071b45 100644 --- a/configuration/ldap/ldap_config.py +++ b/configuration/ldap/ldap_config.py @@ -1,9 +1,10 @@ -import ldap - -from django_auth_ldap.config import LDAPSearch from importlib import import_module from os import environ +import ldap +from django_auth_ldap.config import LDAPSearch + + # Read secret from file def _read_secret(secret_name, default=None): try: @@ -47,9 +48,11 @@ LDAP_IGNORE_CERT_ERRORS = environ.get('LDAP_IGNORE_CERT_ERRORS', 'False').lower( AUTH_LDAP_USER_SEARCH_BASEDN = environ.get('AUTH_LDAP_USER_SEARCH_BASEDN', '') AUTH_LDAP_USER_SEARCH_ATTR = environ.get('AUTH_LDAP_USER_SEARCH_ATTR', 'sAMAccountName') -AUTH_LDAP_USER_SEARCH = LDAPSearch(AUTH_LDAP_USER_SEARCH_BASEDN, - ldap.SCOPE_SUBTREE, - "(" + AUTH_LDAP_USER_SEARCH_ATTR + "=%(user)s)") +AUTH_LDAP_USER_SEARCH = LDAPSearch( + AUTH_LDAP_USER_SEARCH_BASEDN, + ldap.SCOPE_SUBTREE, + "(" + AUTH_LDAP_USER_SEARCH_ATTR + "=%(user)s)" +) # This search ought to return all groups to which the user belongs. django_auth_ldap uses this to determine group # heirarchy. diff --git a/docker/configuration.docker.py b/docker/configuration.docker.py index 0bd4d51..3cd5f7b 100644 --- a/docker/configuration.docker.py +++ b/docker/configuration.docker.py @@ -4,58 +4,58 @@ # # They can be imported by other code (see `ldap_config.py` for an example). -from os.path import abspath, isfile -from os import scandir import importlib.util import sys +from os import scandir +from os.path import abspath, isfile def _filename(f): - return f.name + return f.name def _import(module_name, path, loaded_configurations): - spec = importlib.util.spec_from_file_location('', path) - module = importlib.util.module_from_spec(spec) - spec.loader.exec_module(module) - sys.modules[module_name] = module + spec = importlib.util.spec_from_file_location("", path) + module = importlib.util.module_from_spec(spec) + spec.loader.exec_module(module) + sys.modules[module_name] = module - loaded_configurations.insert(0, module) + loaded_configurations.insert(0, module) - print(f"🧬 loaded config '{path}'") + print(f"🧬 loaded config '{path}'") def read_configurations(config_module, config_dir, main_config): - loaded_configurations = [] + loaded_configurations = [] - main_config_path = abspath(f'{config_dir}/{main_config}.py') - if isfile(main_config_path): - _import(f'{config_module}.{main_config}', main_config_path, loaded_configurations) - else: - print(f"⚠️ Main configuration '{main_config_path}' not found.") + main_config_path = abspath(f"{config_dir}/{main_config}.py") + if isfile(main_config_path): + _import(f"{config_module}.{main_config}", main_config_path, loaded_configurations) + else: + print(f"⚠️ Main configuration '{main_config_path}' not found.") - with scandir(config_dir) as it: - for f in sorted(it, key=_filename): - if not f.is_file(): - continue + with scandir(config_dir) as it: + for f in sorted(it, key=_filename): + if not f.is_file(): + continue - if f.name.startswith('__'): - continue + if f.name.startswith("__"): + continue - if not f.name.endswith('.py'): - continue + if not f.name.endswith(".py"): + continue - if f.name == f'{config_dir}.py': - continue + if f.name == f"{config_dir}.py": + continue - module_name = f"{config_module}.{f.name[:-len('.py')]}".replace(".", "_") - _import(module_name, f.path, loaded_configurations) + module_name = f"{config_module}.{f.name[:-len('.py')]}".replace(".", "_") + _import(module_name, f.path, loaded_configurations) - if len(loaded_configurations) == 0: - print(f"‼️ No configuration files found in '{config_dir}'.") - raise ImportError(f"No configuration files found in '{config_dir}'.") + if len(loaded_configurations) == 0: + print(f"‼️ No configuration files found in '{config_dir}'.") + raise ImportError(f"No configuration files found in '{config_dir}'.") - return loaded_configurations + return loaded_configurations ## Specific Parts @@ -66,15 +66,16 @@ def read_configurations(config_module, config_dir, main_config): _loaded_configurations = read_configurations( - config_dir = '/etc/netbox/config/', - config_module = 'netbox.configuration', - main_config = 'configuration') + config_dir="/etc/netbox/config/", + config_module="netbox.configuration", + main_config="configuration", +) def __getattr__(name): - for config in _loaded_configurations: - try: - return getattr(config, name) - except: - pass - raise AttributeError + for config in _loaded_configurations: + try: + return getattr(config, name) + except: + pass + raise AttributeError diff --git a/docker/ldap_config.docker.py b/docker/ldap_config.docker.py index 169007f..e2509fe 100644 --- a/docker/ldap_config.docker.py +++ b/docker/ldap_config.docker.py @@ -1,21 +1,23 @@ from .configuration import read_configurations _loaded_configurations = read_configurations( - config_dir = '/etc/netbox/config/ldap/', - config_module = 'netbox.configuration.ldap', - main_config = 'ldap_config') + config_dir="/etc/netbox/config/ldap/", + config_module="netbox.configuration.ldap", + main_config="ldap_config", +) def __getattr__(name): - for config in _loaded_configurations: - try: - return getattr(config, name) - except: - pass - raise AttributeError + for config in _loaded_configurations: + try: + return getattr(config, name) + except: + pass + raise AttributeError + def __dir__(): - names = [] - for config in _loaded_configurations: - names.extend(config.__dir__()) - return names + names = [] + for config in _loaded_configurations: + names.extend(config.__dir__()) + return names diff --git a/initializers/custom_fields.yml b/initializers/custom_fields.yml index 654fb69..beed68b 100644 --- a/initializers/custom_fields.yml +++ b/initializers/custom_fields.yml @@ -53,20 +53,20 @@ # - Fifth Item # - Fourth Item # select_field_legacy_format: -# type: select -# label: Choose between items -# required: false -# filter_logic: loose -# weight: 30 -# on_objects: -# - dcim.models.Device -# choices: -# - value: A # this is the deprecated format. -# - value: B # we only use it for the tests. -# - value: C # please see above for the new format. -# - value: "D like deprecated" -# weight: 999 -# - value: E +# type: select +# label: Choose between items +# required: false +# filter_logic: loose +# weight: 30 +# on_objects: +# - dcim.models.Device +# choices: +# - value: A # this is the deprecated format. +# - value: B # we only use it for the tests. +# - value: C # please see above for the new format. +# - value: "D like deprecated" +# weight: 999 +# - value: E # boolean_field: # type: boolean # label: Yes Or No? diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..3b48bae --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,21 @@ +[tool.black] +line-length = 100 +target-version = ['py38'] +include = '\.pyi?$' +exclude = ''' + +( + /( + \.eggs # exclude a few common directories in the + | \.git # root of the project + | \.venv + | \.netbox + | \.vscode + | configuration + )/ +) +''' + +[tool.isort] +profile = "black" +multi_line_output = 3 diff --git a/startup_scripts/000_users.py b/startup_scripts/000_users.py index ffd4bec..66b8519 100644 --- a/startup_scripts/000_users.py +++ b/startup_scripts/000_users.py @@ -4,20 +4,21 @@ from django.contrib.auth.models import User from startup_script_utils import load_yaml, set_permissions from users.models import Token -users = load_yaml('/opt/netbox/initializers/users.yml') +users = load_yaml("/opt/netbox/initializers/users.yml") if users is None: - sys.exit() + sys.exit() for username, user_details in users.items(): - if not User.objects.filter(username=username): - user = User.objects.create_user( - username = username, - password = user_details.get('password', 0) or User.objects.make_random_password()) + if not User.objects.filter(username=username): + user = User.objects.create_user( + username=username, + password=user_details.get("password", 0) or User.objects.make_random_password(), + ) - print("👤 Created user",username) + print("👤 Created user", username) - if user_details.get('api_token', 0): - Token.objects.create(user=user, key=user_details['api_token']) + if user_details.get("api_token", 0): + Token.objects.create(user=user, key=user_details["api_token"]) - yaml_permissions = user_details.get('permissions', []) - set_permissions(user.user_permissions, yaml_permissions) + yaml_permissions = user_details.get("permissions", []) + set_permissions(user.user_permissions, yaml_permissions) diff --git a/startup_scripts/010_groups.py b/startup_scripts/010_groups.py index 951ca96..6726868 100644 --- a/startup_scripts/010_groups.py +++ b/startup_scripts/010_groups.py @@ -3,21 +3,21 @@ import sys from django.contrib.auth.models import Group, User from startup_script_utils import load_yaml, set_permissions -groups = load_yaml('/opt/netbox/initializers/groups.yml') +groups = load_yaml("/opt/netbox/initializers/groups.yml") if groups is None: - sys.exit() + sys.exit() for groupname, group_details in groups.items(): - group, created = Group.objects.get_or_create(name=groupname) + group, created = Group.objects.get_or_create(name=groupname) - if created: - print("👥 Created group", groupname) + if created: + print("👥 Created group", groupname) - for username in group_details.get('users', []): - user = User.objects.get(username=username) + for username in group_details.get("users", []): + user = User.objects.get(username=username) - if user: - user.groups.add(group) + if user: + user.groups.add(group) - yaml_permissions = group_details.get('permissions', []) - set_permissions(group.permissions, yaml_permissions) + yaml_permissions = group_details.get("permissions", []) + set_permissions(group.permissions, yaml_permissions) diff --git a/startup_scripts/020_custom_fields.py b/startup_scripts/020_custom_fields.py index 7479a5e..64566bd 100644 --- a/startup_scripts/020_custom_fields.py +++ b/startup_scripts/020_custom_fields.py @@ -3,56 +3,62 @@ import sys from extras.models import CustomField from startup_script_utils import load_yaml + def get_class_for_class_path(class_path): - import importlib - from django.contrib.contenttypes.models import ContentType + import importlib - module_name, class_name = class_path.rsplit(".", 1) - module = importlib.import_module(module_name) - clazz = getattr(module, class_name) - return ContentType.objects.get_for_model(clazz) + from django.contrib.contenttypes.models import ContentType -customfields = load_yaml('/opt/netbox/initializers/custom_fields.yml') + module_name, class_name = class_path.rsplit(".", 1) + module = importlib.import_module(module_name) + clazz = getattr(module, class_name) + return ContentType.objects.get_for_model(clazz) + + +customfields = load_yaml("/opt/netbox/initializers/custom_fields.yml") if customfields is None: - sys.exit() + sys.exit() for cf_name, cf_details in customfields.items(): - custom_field, created = CustomField.objects.get_or_create(name = cf_name) + custom_field, created = CustomField.objects.get_or_create(name=cf_name) - if created: - if cf_details.get('default', False): - custom_field.default = cf_details['default'] + if created: + if cf_details.get("default", False): + custom_field.default = cf_details["default"] - if cf_details.get('description', False): - custom_field.description = cf_details['description'] + if cf_details.get("description", False): + custom_field.description = cf_details["description"] - if cf_details.get('label', False): - custom_field.label = cf_details['label'] + if cf_details.get("label", False): + custom_field.label = cf_details["label"] - for object_type in cf_details.get('on_objects', []): - custom_field.content_types.add(get_class_for_class_path(object_type)) + for object_type in cf_details.get("on_objects", []): + custom_field.content_types.add(get_class_for_class_path(object_type)) - if cf_details.get('required', False): - custom_field.required = cf_details['required'] + if cf_details.get("required", False): + custom_field.required = cf_details["required"] - if cf_details.get('type', False): - custom_field.type = cf_details['type'] + if cf_details.get("type", False): + custom_field.type = cf_details["type"] - if cf_details.get('weight', -1) >= 0: - custom_field.weight = cf_details['weight'] + if cf_details.get("weight", -1) >= 0: + custom_field.weight = cf_details["weight"] - if cf_details.get('choices', False): - custom_field.choices = [] + if cf_details.get("choices", False): + custom_field.choices = [] - for choice_detail in cf_details.get('choices', []): - if isinstance(choice_detail, dict) and 'value' in choice_detail: - # legacy mode - print(f"⚠️ Please migrate the choice '{choice_detail['value']}' of '{cf_name}' to the new format, as 'weight' is no longer supported!") - custom_field.choices.append(choice_detail['value']) - else: - custom_field.choices.append(choice_detail) + for choice_detail in cf_details.get("choices", []): + if isinstance(choice_detail, dict) and "value" in choice_detail: + # legacy mode + print( + f"⚠️ Please migrate the choice '{choice_detail['value']}' of '{cf_name}'" + + " to the new format, as 'weight' is no longer supported!" + ) + custom_field.choices.append(choice_detail["value"]) + else: + custom_field.choices.append(choice_detail) - custom_field.save() + custom_field.save() - print("🔧 Created custom field", cf_name) + print("🔧 Created custom field", cf_name) diff --git a/startup_scripts/020_tags.py b/startup_scripts/020_tags.py index 2939e75..e50a000 100644 --- a/startup_scripts/020_tags.py +++ b/startup_scripts/020_tags.py @@ -1,23 +1,23 @@ -from extras.models import Tag -from utilities.choices import ColorChoices - -from startup_script_utils import load_yaml import sys -tags = load_yaml('/opt/netbox/initializers/tags.yml') +from extras.models import Tag +from startup_script_utils import load_yaml +from utilities.choices import ColorChoices + +tags = load_yaml("/opt/netbox/initializers/tags.yml") if tags is None: - sys.exit() + sys.exit() for params in tags: - if 'color' in params: - color = params.pop('color') + if "color" in params: + color = params.pop("color") - for color_tpl in ColorChoices: - if color in color_tpl: - params['color'] = color_tpl[0] + for color_tpl in ColorChoices: + if color in color_tpl: + params["color"] = color_tpl[0] - tag, created = Tag.objects.get_or_create(**params) + tag, created = Tag.objects.get_or_create(**params) - if created: - print("🎨 Created Tag", tag.name) + if created: + print("🎨 Created Tag", tag.name) diff --git a/startup_scripts/030_regions.py b/startup_scripts/030_regions.py index 0b61d89..9d5c91f 100644 --- a/startup_scripts/030_regions.py +++ b/startup_scripts/030_regions.py @@ -1,26 +1,25 @@ -from dcim.models import Region -from startup_script_utils import load_yaml import sys -regions = load_yaml('/opt/netbox/initializers/regions.yml') +from dcim.models import Region +from startup_script_utils import load_yaml + +regions = load_yaml("/opt/netbox/initializers/regions.yml") if regions is None: - sys.exit() + sys.exit() -optional_assocs = { - 'parent': (Region, 'name') -} +optional_assocs = {"parent": (Region, "name")} for params in regions: - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - region, created = Region.objects.get_or_create(**params) + region, created = Region.objects.get_or_create(**params) - if created: - print("🌐 Created region", region.name) + if created: + print("🌐 Created region", region.name) diff --git a/startup_scripts/040_sites.py b/startup_scripts/040_sites.py index a4f7e71..f785139 100644 --- a/startup_scripts/040_sites.py +++ b/startup_scripts/040_sites.py @@ -1,32 +1,29 @@ import sys from dcim.models import Region, Site -from startup_script_utils import * +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant -sites = load_yaml('/opt/netbox/initializers/sites.yml') +sites = load_yaml("/opt/netbox/initializers/sites.yml") if sites is None: - sys.exit() + sys.exit() -optional_assocs = { - 'region': (Region, 'name'), - 'tenant': (Tenant, 'name') -} +optional_assocs = {"region": (Region, "name"), "tenant": (Tenant, "name")} for params in sites: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - site, created = Site.objects.get_or_create(**params) + site, created = Site.objects.get_or_create(**params) - if created: - set_custom_fields_values(site, custom_field_data) + if created: + set_custom_fields_values(site, custom_field_data) - print("📍 Created site", site.name) + print("📍 Created site", site.name) diff --git a/startup_scripts/050_manufacturers.py b/startup_scripts/050_manufacturers.py index b9ebb32..d11b440 100644 --- a/startup_scripts/050_manufacturers.py +++ b/startup_scripts/050_manufacturers.py @@ -1,14 +1,15 @@ -from dcim.models import Manufacturer -from startup_script_utils import load_yaml import sys -manufacturers = load_yaml('/opt/netbox/initializers/manufacturers.yml') +from dcim.models import Manufacturer +from startup_script_utils import load_yaml + +manufacturers = load_yaml("/opt/netbox/initializers/manufacturers.yml") if manufacturers is None: - sys.exit() + sys.exit() for params in manufacturers: - manufacturer, created = Manufacturer.objects.get_or_create(**params) + manufacturer, created = Manufacturer.objects.get_or_create(**params) - if created: - print("🏭 Created Manufacturer", manufacturer.name) + if created: + print("🏭 Created Manufacturer", manufacturer.name) diff --git a/startup_scripts/060_device_types.py b/startup_scripts/060_device_types.py index e0695b1..0d3050b 100644 --- a/startup_scripts/060_device_types.py +++ b/startup_scripts/060_device_types.py @@ -1,42 +1,37 @@ import sys from dcim.models import DeviceType, Manufacturer, Region -from startup_script_utils import * +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant -device_types = load_yaml('/opt/netbox/initializers/device_types.yml') +device_types = load_yaml("/opt/netbox/initializers/device_types.yml") if device_types is None: - sys.exit() + sys.exit() -required_assocs = { - 'manufacturer': (Manufacturer, 'name') -} +required_assocs = {"manufacturer": (Manufacturer, "name")} -optional_assocs = { - 'region': (Region, 'name'), - 'tenant': (Tenant, 'name') -} +optional_assocs = {"region": (Region, "name"), "tenant": (Tenant, "name")} for params in device_types: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - device_type, created = DeviceType.objects.get_or_create(**params) + device_type, created = DeviceType.objects.get_or_create(**params) - if created: - set_custom_fields_values(device_type, custom_field_data) + if created: + set_custom_fields_values(device_type, custom_field_data) - print("🔡 Created device type", device_type.manufacturer, device_type.model) + print("🔡 Created device type", device_type.manufacturer, device_type.model) diff --git a/startup_scripts/070_rack_roles.py b/startup_scripts/070_rack_roles.py index f5aa2ce..5850405 100644 --- a/startup_scripts/070_rack_roles.py +++ b/startup_scripts/070_rack_roles.py @@ -1,23 +1,23 @@ -from dcim.models import RackRole -from utilities.choices import ColorChoices - -from startup_script_utils import load_yaml import sys -rack_roles = load_yaml('/opt/netbox/initializers/rack_roles.yml') +from dcim.models import RackRole +from startup_script_utils import load_yaml +from utilities.choices import ColorChoices + +rack_roles = load_yaml("/opt/netbox/initializers/rack_roles.yml") if rack_roles is None: - sys.exit() + sys.exit() for params in rack_roles: - if 'color' in params: - color = params.pop('color') + if "color" in params: + color = params.pop("color") - for color_tpl in ColorChoices: - if color in color_tpl: - params['color'] = color_tpl[0] + for color_tpl in ColorChoices: + if color in color_tpl: + params["color"] = color_tpl[0] - rack_role, created = RackRole.objects.get_or_create(**params) + rack_role, created = RackRole.objects.get_or_create(**params) - if created: - print("🎨 Created rack role", rack_role.name) + if created: + print("🎨 Created rack role", rack_role.name) diff --git a/startup_scripts/075_rack_groups.py b/startup_scripts/075_rack_groups.py index dc5ec20..3974b56 100644 --- a/startup_scripts/075_rack_groups.py +++ b/startup_scripts/075_rack_groups.py @@ -1,25 +1,23 @@ -from dcim.models import Site,RackGroup -from startup_script_utils import load_yaml import sys -rack_groups = load_yaml('/opt/netbox/initializers/rack_groups.yml') +from dcim.models import RackGroup, Site +from startup_script_utils import load_yaml + +rack_groups = load_yaml("/opt/netbox/initializers/rack_groups.yml") if rack_groups is None: - sys.exit() + sys.exit() -required_assocs = { - 'site': (Site, 'name') -} +required_assocs = {"site": (Site, "name")} for params in rack_groups: - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } - params[assoc] = model.objects.get(**query) + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} + params[assoc] = model.objects.get(**query) - rack_group, created = RackGroup.objects.get_or_create(**params) - - if created: - print("🎨 Created rack group", rack_group.name) + rack_group, created = RackGroup.objects.get_or_create(**params) + if created: + print("🎨 Created rack group", rack_group.name) diff --git a/startup_scripts/080_racks.py b/startup_scripts/080_racks.py index 4504624..087b3f9 100644 --- a/startup_scripts/080_racks.py +++ b/startup_scripts/080_racks.py @@ -1,43 +1,41 @@ import sys -from dcim.models import Site, RackRole, Rack, RackGroup -from startup_script_utils import * +from dcim.models import Rack, RackGroup, RackRole, Site +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant -racks = load_yaml('/opt/netbox/initializers/racks.yml') +racks = load_yaml("/opt/netbox/initializers/racks.yml") if racks is None: - sys.exit() + sys.exit() -required_assocs = { - 'site': (Site, 'name') -} +required_assocs = {"site": (Site, "name")} optional_assocs = { - 'role': (RackRole, 'name'), - 'tenant': (Tenant, 'name'), - 'group': (RackGroup, 'name') + "role": (RackRole, "name"), + "tenant": (Tenant, "name"), + "group": (RackGroup, "name"), } for params in racks: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - rack, created = Rack.objects.get_or_create(**params) + rack, created = Rack.objects.get_or_create(**params) - if created: - set_custom_fields_values(rack, custom_field_data) + if created: + set_custom_fields_values(rack, custom_field_data) - print("🔳 Created rack", rack.site, rack.name) + print("🔳 Created rack", rack.site, rack.name) diff --git a/startup_scripts/090_device_roles.py b/startup_scripts/090_device_roles.py index 5225cc0..635acff 100644 --- a/startup_scripts/090_device_roles.py +++ b/startup_scripts/090_device_roles.py @@ -1,24 +1,24 @@ -from dcim.models import DeviceRole -from utilities.choices import ColorChoices - -from startup_script_utils import load_yaml import sys -device_roles = load_yaml('/opt/netbox/initializers/device_roles.yml') +from dcim.models import DeviceRole +from startup_script_utils import load_yaml +from utilities.choices import ColorChoices + +device_roles = load_yaml("/opt/netbox/initializers/device_roles.yml") if device_roles is None: - sys.exit() + sys.exit() for params in device_roles: - if 'color' in params: - color = params.pop('color') + if "color" in params: + color = params.pop("color") - for color_tpl in ColorChoices: - if color in color_tpl: - params['color'] = color_tpl[0] + for color_tpl in ColorChoices: + if color in color_tpl: + params["color"] = color_tpl[0] - device_role, created = DeviceRole.objects.get_or_create(**params) + device_role, created = DeviceRole.objects.get_or_create(**params) - if created: - print("🎨 Created device role", device_role.name) + if created: + print("🎨 Created device role", device_role.name) diff --git a/startup_scripts/100_platforms.py b/startup_scripts/100_platforms.py index 3673230..633b89f 100644 --- a/startup_scripts/100_platforms.py +++ b/startup_scripts/100_platforms.py @@ -1,26 +1,27 @@ -from dcim.models import Manufacturer, Platform -from startup_script_utils import load_yaml import sys -platforms = load_yaml('/opt/netbox/initializers/platforms.yml') +from dcim.models import Manufacturer, Platform +from startup_script_utils import load_yaml + +platforms = load_yaml("/opt/netbox/initializers/platforms.yml") if platforms is None: - sys.exit() + sys.exit() optional_assocs = { - 'manufacturer': (Manufacturer, 'name'), + "manufacturer": (Manufacturer, "name"), } for params in platforms: - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - platform, created = Platform.objects.get_or_create(**params) + platform, created = Platform.objects.get_or_create(**params) - if created: - print("💾 Created platform", platform.name) + if created: + print("💾 Created platform", platform.name) diff --git a/startup_scripts/110_tenant_groups.py b/startup_scripts/110_tenant_groups.py index c106d67..65cf155 100644 --- a/startup_scripts/110_tenant_groups.py +++ b/startup_scripts/110_tenant_groups.py @@ -1,14 +1,15 @@ -from tenancy.models import TenantGroup -from startup_script_utils import load_yaml import sys -tenant_groups = load_yaml('/opt/netbox/initializers/tenant_groups.yml') +from startup_script_utils import load_yaml +from tenancy.models import TenantGroup + +tenant_groups = load_yaml("/opt/netbox/initializers/tenant_groups.yml") if tenant_groups is None: - sys.exit() + sys.exit() for params in tenant_groups: - tenant_group, created = TenantGroup.objects.get_or_create(**params) + tenant_group, created = TenantGroup.objects.get_or_create(**params) - if created: - print("🔳 Created Tenant Group", tenant_group.name) + if created: + print("🔳 Created Tenant Group", tenant_group.name) diff --git a/startup_scripts/120_tenants.py b/startup_scripts/120_tenants.py index 3b3b221..7b1a629 100644 --- a/startup_scripts/120_tenants.py +++ b/startup_scripts/120_tenants.py @@ -1,30 +1,28 @@ import sys -from startup_script_utils import * +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant, TenantGroup -tenants = load_yaml('/opt/netbox/initializers/tenants.yml') +tenants = load_yaml("/opt/netbox/initializers/tenants.yml") if tenants is None: - sys.exit() + sys.exit() -optional_assocs = { - 'group': (TenantGroup, 'name') -} +optional_assocs = {"group": (TenantGroup, "name")} for params in tenants: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - tenant, created = Tenant.objects.get_or_create(**params) + tenant, created = Tenant.objects.get_or_create(**params) - if created: - set_custom_fields_values(tenant, custom_field_data) + if created: + set_custom_fields_values(tenant, custom_field_data) - print("👩‍💻 Created Tenant", tenant.name) + print("👩‍💻 Created Tenant", tenant.name) diff --git a/startup_scripts/130_cluster_types.py b/startup_scripts/130_cluster_types.py index d39933f..9f361b1 100644 --- a/startup_scripts/130_cluster_types.py +++ b/startup_scripts/130_cluster_types.py @@ -1,14 +1,15 @@ -from virtualization.models import ClusterType -from startup_script_utils import load_yaml import sys -cluster_types = load_yaml('/opt/netbox/initializers/cluster_types.yml') +from startup_script_utils import load_yaml +from virtualization.models import ClusterType + +cluster_types = load_yaml("/opt/netbox/initializers/cluster_types.yml") if cluster_types is None: - sys.exit() + sys.exit() for params in cluster_types: - cluster_type, created = ClusterType.objects.get_or_create(**params) + cluster_type, created = ClusterType.objects.get_or_create(**params) - if created: - print("🧰 Created Cluster Type", cluster_type.name) + if created: + print("🧰 Created Cluster Type", cluster_type.name) diff --git a/startup_scripts/135_cluster_groups.py b/startup_scripts/135_cluster_groups.py index b52518a..fedd292 100644 --- a/startup_scripts/135_cluster_groups.py +++ b/startup_scripts/135_cluster_groups.py @@ -1,14 +1,15 @@ -from virtualization.models import ClusterGroup -from startup_script_utils import load_yaml import sys -cluster_groups = load_yaml('/opt/netbox/initializers/cluster_groups.yml') +from startup_script_utils import load_yaml +from virtualization.models import ClusterGroup + +cluster_groups = load_yaml("/opt/netbox/initializers/cluster_groups.yml") if cluster_groups is None: - sys.exit() + sys.exit() for params in cluster_groups: - cluster_group, created = ClusterGroup.objects.get_or_create(**params) + cluster_group, created = ClusterGroup.objects.get_or_create(**params) - if created: - print("🗄️ Created Cluster Group", cluster_group.name) + if created: + print("🗄️ Created Cluster Group", cluster_group.name) diff --git a/startup_scripts/135_clusters.py b/startup_scripts/135_clusters.py index 85e462f..2748f20 100644 --- a/startup_scripts/135_clusters.py +++ b/startup_scripts/135_clusters.py @@ -1,44 +1,42 @@ import sys from dcim.models import Site -from startup_script_utils import * -from virtualization.models import Cluster, ClusterType, ClusterGroup +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant +from virtualization.models import Cluster, ClusterGroup, ClusterType -clusters = load_yaml('/opt/netbox/initializers/clusters.yml') +clusters = load_yaml("/opt/netbox/initializers/clusters.yml") if clusters is None: - sys.exit() + sys.exit() -required_assocs = { - 'type': (ClusterType, 'name') -} +required_assocs = {"type": (ClusterType, "name")} optional_assocs = { - 'site': (Site, 'name'), - 'group': (ClusterGroup, 'name'), - 'tenant': (Tenant, 'name') + "site": (Site, "name"), + "group": (ClusterGroup, "name"), + "tenant": (Tenant, "name"), } for params in clusters: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - cluster, created = Cluster.objects.get_or_create(**params) + cluster, created = Cluster.objects.get_or_create(**params) - if created: - set_custom_fields_values(cluster, custom_field_data) + if created: + set_custom_fields_values(cluster, custom_field_data) - print("🗄️ Created cluster", cluster.name) + print("🗄️ Created cluster", cluster.name) diff --git a/startup_scripts/140_clusters.py b/startup_scripts/140_clusters.py index 85e462f..2748f20 100644 --- a/startup_scripts/140_clusters.py +++ b/startup_scripts/140_clusters.py @@ -1,44 +1,42 @@ import sys from dcim.models import Site -from startup_script_utils import * -from virtualization.models import Cluster, ClusterType, ClusterGroup +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant +from virtualization.models import Cluster, ClusterGroup, ClusterType -clusters = load_yaml('/opt/netbox/initializers/clusters.yml') +clusters = load_yaml("/opt/netbox/initializers/clusters.yml") if clusters is None: - sys.exit() + sys.exit() -required_assocs = { - 'type': (ClusterType, 'name') -} +required_assocs = {"type": (ClusterType, "name")} optional_assocs = { - 'site': (Site, 'name'), - 'group': (ClusterGroup, 'name'), - 'tenant': (Tenant, 'name') + "site": (Site, "name"), + "group": (ClusterGroup, "name"), + "tenant": (Tenant, "name"), } for params in clusters: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - cluster, created = Cluster.objects.get_or_create(**params) + cluster, created = Cluster.objects.get_or_create(**params) - if created: - set_custom_fields_values(cluster, custom_field_data) + if created: + set_custom_fields_values(cluster, custom_field_data) - print("🗄️ Created cluster", cluster.name) + print("🗄️ Created cluster", cluster.name) diff --git a/startup_scripts/140_devices.py b/startup_scripts/140_devices.py index d73fd07..597db0c 100644 --- a/startup_scripts/140_devices.py +++ b/startup_scripts/140_devices.py @@ -1,51 +1,51 @@ import sys -from dcim.models import Site, Rack, DeviceRole, DeviceType, Device, Platform -from startup_script_utils import * +from dcim.models import Device, DeviceRole, DeviceType, Platform, Rack, Site +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant from virtualization.models import Cluster -devices = load_yaml('/opt/netbox/initializers/devices.yml') +devices = load_yaml("/opt/netbox/initializers/devices.yml") if devices is None: - sys.exit() + sys.exit() required_assocs = { - 'device_role': (DeviceRole, 'name'), - 'device_type': (DeviceType, 'model'), - 'site': (Site, 'name') + "device_role": (DeviceRole, "name"), + "device_type": (DeviceType, "model"), + "site": (Site, "name"), } optional_assocs = { - 'tenant': (Tenant, 'name'), - 'platform': (Platform, 'name'), - 'rack': (Rack, 'name'), - 'cluster': (Cluster, 'name') + "tenant": (Tenant, "name"), + "platform": (Platform, "name"), + "rack": (Rack, "name"), + "cluster": (Cluster, "name"), } for params in devices: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - # primary ips are handled later in `270_primary_ips.py` - params.pop('primary_ip4', None) - params.pop('primary_ip6', None) + # primary ips are handled later in `270_primary_ips.py` + params.pop("primary_ip4", None) + params.pop("primary_ip6", None) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - device, created = Device.objects.get_or_create(**params) + device, created = Device.objects.get_or_create(**params) - if created: - set_custom_fields_values(device, custom_field_data) + if created: + set_custom_fields_values(device, custom_field_data) - print("🖥️ Created device", device.name) + print("🖥️ Created device", device.name) diff --git a/startup_scripts/145_devices.py b/startup_scripts/145_devices.py index d73fd07..597db0c 100644 --- a/startup_scripts/145_devices.py +++ b/startup_scripts/145_devices.py @@ -1,51 +1,51 @@ import sys -from dcim.models import Site, Rack, DeviceRole, DeviceType, Device, Platform -from startup_script_utils import * +from dcim.models import Device, DeviceRole, DeviceType, Platform, Rack, Site +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant from virtualization.models import Cluster -devices = load_yaml('/opt/netbox/initializers/devices.yml') +devices = load_yaml("/opt/netbox/initializers/devices.yml") if devices is None: - sys.exit() + sys.exit() required_assocs = { - 'device_role': (DeviceRole, 'name'), - 'device_type': (DeviceType, 'model'), - 'site': (Site, 'name') + "device_role": (DeviceRole, "name"), + "device_type": (DeviceType, "model"), + "site": (Site, "name"), } optional_assocs = { - 'tenant': (Tenant, 'name'), - 'platform': (Platform, 'name'), - 'rack': (Rack, 'name'), - 'cluster': (Cluster, 'name') + "tenant": (Tenant, "name"), + "platform": (Platform, "name"), + "rack": (Rack, "name"), + "cluster": (Cluster, "name"), } for params in devices: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - # primary ips are handled later in `270_primary_ips.py` - params.pop('primary_ip4', None) - params.pop('primary_ip6', None) + # primary ips are handled later in `270_primary_ips.py` + params.pop("primary_ip4", None) + params.pop("primary_ip6", None) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - device, created = Device.objects.get_or_create(**params) + device, created = Device.objects.get_or_create(**params) - if created: - set_custom_fields_values(device, custom_field_data) + if created: + set_custom_fields_values(device, custom_field_data) - print("🖥️ Created device", device.name) + print("🖥️ Created device", device.name) diff --git a/startup_scripts/150_rirs.py b/startup_scripts/150_rirs.py index 8bcf51f..0e0df20 100644 --- a/startup_scripts/150_rirs.py +++ b/startup_scripts/150_rirs.py @@ -1,14 +1,15 @@ -from ipam.models import RIR -from startup_script_utils import load_yaml import sys -rirs = load_yaml('/opt/netbox/initializers/rirs.yml') +from ipam.models import RIR +from startup_script_utils import load_yaml + +rirs = load_yaml("/opt/netbox/initializers/rirs.yml") if rirs is None: - sys.exit() + sys.exit() for params in rirs: - rir, created = RIR.objects.get_or_create(**params) + rir, created = RIR.objects.get_or_create(**params) - if created: - print("🗺️ Created RIR", rir.name) + if created: + print("🗺️ Created RIR", rir.name) diff --git a/startup_scripts/160_aggregates.py b/startup_scripts/160_aggregates.py index 40aea1b..c638e6f 100644 --- a/startup_scripts/160_aggregates.py +++ b/startup_scripts/160_aggregates.py @@ -1,44 +1,42 @@ import sys -from ipam.models import Aggregate, RIR +from ipam.models import RIR, Aggregate from netaddr import IPNetwork -from startup_script_utils import * +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant -aggregates = load_yaml('/opt/netbox/initializers/aggregates.yml') +aggregates = load_yaml("/opt/netbox/initializers/aggregates.yml") if aggregates is None: - sys.exit() + sys.exit() -required_assocs = { - 'rir': (RIR, 'name') -} +required_assocs = {"rir": (RIR, "name")} optional_assocs = { - 'tenant': (Tenant, 'name'), + "tenant": (Tenant, "name"), } for params in aggregates: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - params['prefix'] = IPNetwork(params['prefix']) + params["prefix"] = IPNetwork(params["prefix"]) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - aggregate, created = Aggregate.objects.get_or_create(**params) + aggregate, created = Aggregate.objects.get_or_create(**params) - if created: - set_custom_fields_values(aggregate, custom_field_data) + if created: + set_custom_fields_values(aggregate, custom_field_data) - print("🗞️ Created Aggregate", aggregate.prefix) + print("🗞️ Created Aggregate", aggregate.prefix) diff --git a/startup_scripts/165_cluster_groups.py b/startup_scripts/165_cluster_groups.py index b52518a..fedd292 100644 --- a/startup_scripts/165_cluster_groups.py +++ b/startup_scripts/165_cluster_groups.py @@ -1,14 +1,15 @@ -from virtualization.models import ClusterGroup -from startup_script_utils import load_yaml import sys -cluster_groups = load_yaml('/opt/netbox/initializers/cluster_groups.yml') +from startup_script_utils import load_yaml +from virtualization.models import ClusterGroup + +cluster_groups = load_yaml("/opt/netbox/initializers/cluster_groups.yml") if cluster_groups is None: - sys.exit() + sys.exit() for params in cluster_groups: - cluster_group, created = ClusterGroup.objects.get_or_create(**params) + cluster_group, created = ClusterGroup.objects.get_or_create(**params) - if created: - print("🗄️ Created Cluster Group", cluster_group.name) + if created: + print("🗄️ Created Cluster Group", cluster_group.name) diff --git a/startup_scripts/175_route_targets.py b/startup_scripts/175_route_targets.py index efdaba6..e1c8221 100644 --- a/startup_scripts/175_route_targets.py +++ b/startup_scripts/175_route_targets.py @@ -1,31 +1,29 @@ import sys from ipam.models import RouteTarget -from startup_script_utils import * +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant -route_targets = load_yaml('/opt/netbox/initializers/route_targets.yml') +route_targets = load_yaml("/opt/netbox/initializers/route_targets.yml") if route_targets is None: - sys.exit() + sys.exit() -optional_assocs = { - 'tenant': (Tenant, 'name') -} +optional_assocs = {"tenant": (Tenant, "name")} for params in route_targets: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - route_target, created = RouteTarget.objects.get_or_create(**params) + route_target, created = RouteTarget.objects.get_or_create(**params) - if created: - set_custom_fields_values(route_target, custom_field_data) + if created: + set_custom_fields_values(route_target, custom_field_data) - print("🎯 Created Route Target", route_target.name) + print("🎯 Created Route Target", route_target.name) diff --git a/startup_scripts/180_vrfs.py b/startup_scripts/180_vrfs.py index 2f22316..a67c8c8 100644 --- a/startup_scripts/180_vrfs.py +++ b/startup_scripts/180_vrfs.py @@ -1,31 +1,29 @@ import sys from ipam.models import VRF -from startup_script_utils import * +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant -vrfs = load_yaml('/opt/netbox/initializers/vrfs.yml') +vrfs = load_yaml("/opt/netbox/initializers/vrfs.yml") if vrfs is None: - sys.exit() + sys.exit() -optional_assocs = { - 'tenant': (Tenant, 'name') -} +optional_assocs = {"tenant": (Tenant, "name")} for params in vrfs: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - vrf, created = VRF.objects.get_or_create(**params) + vrf, created = VRF.objects.get_or_create(**params) - if created: - set_custom_fields_values(vrf, custom_field_data) + if created: + set_custom_fields_values(vrf, custom_field_data) - print("📦 Created VRF", vrf.name) + print("📦 Created VRF", vrf.name) diff --git a/startup_scripts/190_prefix_vlan_roles.py b/startup_scripts/190_prefix_vlan_roles.py index 72c8eee..ec359fb 100644 --- a/startup_scripts/190_prefix_vlan_roles.py +++ b/startup_scripts/190_prefix_vlan_roles.py @@ -1,14 +1,15 @@ -from ipam.models import Role -from startup_script_utils import load_yaml import sys -roles = load_yaml('/opt/netbox/initializers/prefix_vlan_roles.yml') +from ipam.models import Role +from startup_script_utils import load_yaml + +roles = load_yaml("/opt/netbox/initializers/prefix_vlan_roles.yml") if roles is None: - sys.exit() + sys.exit() for params in roles: - role, created = Role.objects.get_or_create(**params) + role, created = Role.objects.get_or_create(**params) - if created: - print("⛹️‍ Created Prefix/VLAN Role", role.name) + if created: + print("⛹️‍ Created Prefix/VLAN Role", role.name) diff --git a/startup_scripts/200_vlan_groups.py b/startup_scripts/200_vlan_groups.py index 35c3616..7b72b2d 100644 --- a/startup_scripts/200_vlan_groups.py +++ b/startup_scripts/200_vlan_groups.py @@ -2,30 +2,28 @@ import sys from dcim.models import Site from ipam.models import VLANGroup -from startup_script_utils import * +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values -vlan_groups = load_yaml('/opt/netbox/initializers/vlan_groups.yml') +vlan_groups = load_yaml("/opt/netbox/initializers/vlan_groups.yml") if vlan_groups is None: - sys.exit() + sys.exit() -optional_assocs = { - 'site': (Site, 'name') -} +optional_assocs = {"site": (Site, "name")} for params in vlan_groups: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - vlan_group, created = VLANGroup.objects.get_or_create(**params) + vlan_group, created = VLANGroup.objects.get_or_create(**params) - if created: - set_custom_fields_values(vlan_group, custom_field_data) + if created: + set_custom_fields_values(vlan_group, custom_field_data) - print("🏘️ Created VLAN Group", vlan_group.name) + print("🏘️ Created VLAN Group", vlan_group.name) diff --git a/startup_scripts/210_vlans.py b/startup_scripts/210_vlans.py index 7848fc1..e8ebb94 100644 --- a/startup_scripts/210_vlans.py +++ b/startup_scripts/210_vlans.py @@ -1,36 +1,36 @@ import sys from dcim.models import Site -from ipam.models import VLAN, VLANGroup, Role -from startup_script_utils import * +from ipam.models import VLAN, Role, VLANGroup +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant, TenantGroup -vlans = load_yaml('/opt/netbox/initializers/vlans.yml') +vlans = load_yaml("/opt/netbox/initializers/vlans.yml") if vlans is None: - sys.exit() + sys.exit() optional_assocs = { - 'site': (Site, 'name'), - 'tenant': (Tenant, 'name'), - 'tenant_group': (TenantGroup, 'name'), - 'group': (VLANGroup, 'name'), - 'role': (Role, 'name') + "site": (Site, "name"), + "tenant": (Tenant, "name"), + "tenant_group": (TenantGroup, "name"), + "group": (VLANGroup, "name"), + "role": (Role, "name"), } for params in vlans: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - vlan, created = VLAN.objects.get_or_create(**params) + vlan, created = VLAN.objects.get_or_create(**params) - if created: - set_custom_fields_values(vlan, custom_field_data) + if created: + set_custom_fields_values(vlan, custom_field_data) - print("🏠 Created VLAN", vlan.name) + print("🏠 Created VLAN", vlan.name) diff --git a/startup_scripts/220_prefixes.py b/startup_scripts/220_prefixes.py index 9175521..4e2b0d0 100644 --- a/startup_scripts/220_prefixes.py +++ b/startup_scripts/220_prefixes.py @@ -1,39 +1,39 @@ import sys from dcim.models import Site -from ipam.models import Prefix, VLAN, Role, VRF +from ipam.models import VLAN, VRF, Prefix, Role from netaddr import IPNetwork -from startup_script_utils import * +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant, TenantGroup -prefixes = load_yaml('/opt/netbox/initializers/prefixes.yml') +prefixes = load_yaml("/opt/netbox/initializers/prefixes.yml") if prefixes is None: - sys.exit() + sys.exit() optional_assocs = { - 'site': (Site, 'name'), - 'tenant': (Tenant, 'name'), - 'tenant_group': (TenantGroup, 'name'), - 'vlan': (VLAN, 'name'), - 'role': (Role, 'name'), - 'vrf': (VRF, 'name') + "site": (Site, "name"), + "tenant": (Tenant, "name"), + "tenant_group": (TenantGroup, "name"), + "vlan": (VLAN, "name"), + "role": (Role, "name"), + "vrf": (VRF, "name"), } for params in prefixes: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - params['prefix'] = IPNetwork(params['prefix']) + params["prefix"] = IPNetwork(params["prefix"]) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } - params[assoc] = model.objects.get(**query) + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} + params[assoc] = model.objects.get(**query) - prefix, created = Prefix.objects.get_or_create(**params) + prefix, created = Prefix.objects.get_or_create(**params) - if created: - set_custom_fields_values(prefix, custom_field_data) + if created: + set_custom_fields_values(prefix, custom_field_data) - print("📌 Created Prefix", prefix.prefix) + print("📌 Created Prefix", prefix.prefix) diff --git a/startup_scripts/230_virtual_machines.py b/startup_scripts/230_virtual_machines.py index 328d3ea..2e3f428 100644 --- a/startup_scripts/230_virtual_machines.py +++ b/startup_scripts/230_virtual_machines.py @@ -1,48 +1,46 @@ import sys -from dcim.models import Platform, DeviceRole -from startup_script_utils import * +from dcim.models import DeviceRole, Platform +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant from virtualization.models import Cluster, VirtualMachine -virtual_machines = load_yaml('/opt/netbox/initializers/virtual_machines.yml') +virtual_machines = load_yaml("/opt/netbox/initializers/virtual_machines.yml") if virtual_machines is None: - sys.exit() + sys.exit() -required_assocs = { - 'cluster': (Cluster, 'name') -} +required_assocs = {"cluster": (Cluster, "name")} optional_assocs = { - 'tenant': (Tenant, 'name'), - 'platform': (Platform, 'name'), - 'role': (DeviceRole, 'name') + "tenant": (Tenant, "name"), + "platform": (Platform, "name"), + "role": (DeviceRole, "name"), } for params in virtual_machines: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - # primary ips are handled later in `270_primary_ips.py` - params.pop('primary_ip4', None) - params.pop('primary_ip6', None) + # primary ips are handled later in `270_primary_ips.py` + params.pop("primary_ip4", None) + params.pop("primary_ip6", None) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - virtual_machine, created = VirtualMachine.objects.get_or_create(**params) + virtual_machine, created = VirtualMachine.objects.get_or_create(**params) - if created: - set_custom_fields_values(virtual_machine, custom_field_data) + if created: + set_custom_fields_values(virtual_machine, custom_field_data) - print("🖥️ Created virtual machine", virtual_machine.name) + print("🖥️ Created virtual machine", virtual_machine.name) diff --git a/startup_scripts/240_virtualization_interfaces.py b/startup_scripts/240_virtualization_interfaces.py index c6f8e50..6ee6347 100644 --- a/startup_scripts/240_virtualization_interfaces.py +++ b/startup_scripts/240_virtualization_interfaces.py @@ -1,29 +1,27 @@ import sys -from startup_script_utils import * +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from virtualization.models import VirtualMachine, VMInterface -interfaces = load_yaml('/opt/netbox/initializers/virtualization_interfaces.yml') +interfaces = load_yaml("/opt/netbox/initializers/virtualization_interfaces.yml") if interfaces is None: - sys.exit() + sys.exit() -required_assocs = { - 'virtual_machine': (VirtualMachine, 'name') -} +required_assocs = {"virtual_machine": (VirtualMachine, "name")} for params in interfaces: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - interface, created = VMInterface.objects.get_or_create(**params) + interface, created = VMInterface.objects.get_or_create(**params) - if created: - set_custom_fields_values(interface, custom_field_data) + if created: + set_custom_fields_values(interface, custom_field_data) - print("🧷 Created interface", interface.name, interface.virtual_machine.name) + print("🧷 Created interface", interface.name, interface.virtual_machine.name) diff --git a/startup_scripts/250_dcim_interfaces.py b/startup_scripts/250_dcim_interfaces.py index 3b7067d..a802628 100644 --- a/startup_scripts/250_dcim_interfaces.py +++ b/startup_scripts/250_dcim_interfaces.py @@ -1,29 +1,27 @@ import sys -from dcim.models import Interface, Device -from startup_script_utils import * +from dcim.models import Device, Interface +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values -interfaces = load_yaml('/opt/netbox/initializers/dcim_interfaces.yml') +interfaces = load_yaml("/opt/netbox/initializers/dcim_interfaces.yml") if interfaces is None: - sys.exit() + sys.exit() -required_assocs = { - 'device': (Device, 'name') -} +required_assocs = {"device": (Device, "name")} for params in interfaces: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - interface, created = Interface.objects.get_or_create(**params) + interface, created = Interface.objects.get_or_create(**params) - if created: - set_custom_fields_values(interface, custom_field_data) + if created: + set_custom_fields_values(interface, custom_field_data) - print("🧷 Created interface", interface.name, interface.device.name) + print("🧷 Created interface", interface.name, interface.device.name) diff --git a/startup_scripts/260_ip_addresses.py b/startup_scripts/260_ip_addresses.py index a0582a2..3f0aed0 100644 --- a/startup_scripts/260_ip_addresses.py +++ b/startup_scripts/260_ip_addresses.py @@ -5,56 +5,58 @@ from django.contrib.contenttypes.models import ContentType from django.db.models import Q from ipam.models import VRF, IPAddress from netaddr import IPNetwork -from startup_script_utils import * +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values from tenancy.models import Tenant from virtualization.models import VirtualMachine, VMInterface -ip_addresses = load_yaml('/opt/netbox/initializers/ip_addresses.yml') +ip_addresses = load_yaml("/opt/netbox/initializers/ip_addresses.yml") if ip_addresses is None: - sys.exit() - -optional_assocs = { - 'tenant': (Tenant, 'name'), - 'vrf': (VRF, 'name'), - 'interface': (None, None) -} - -vm_interface_ct = ContentType.objects.filter(Q(app_label='virtualization', model='vminterface')).first() -interface_ct = ContentType.objects.filter(Q(app_label='dcim', model='interface')).first() - -for params in ip_addresses: - custom_field_data = pop_custom_fields(params) - - vm = params.pop('virtual_machine', None) - device = params.pop('device', None) - params['address'] = IPNetwork(params['address']) - - if vm and device: - print("IP Address can only specify one of the following: virtual_machine or device.") sys.exit() - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - if assoc == 'interface': - if vm: - vm_id = VirtualMachine.objects.get(name=vm).id - query = { 'name': params.pop(assoc), "virtual_machine_id": vm_id } - params['assigned_object_type'] = vm_interface_ct - params['assigned_object_id'] = VMInterface.objects.get(**query).id - elif device: - dev_id = Device.objects.get(name=device).id - query = { 'name': params.pop(assoc), "device_id": dev_id } - params['assigned_object_type'] = interface_ct - params['assigned_object_id'] = Interface.objects.get(**query).id - else: - query = { field: params.pop(assoc) } - params[assoc] = model.objects.get(**query) +optional_assocs = { + "tenant": (Tenant, "name"), + "vrf": (VRF, "name"), + "interface": (None, None), +} - ip_address, created = IPAddress.objects.get_or_create(**params) +vm_interface_ct = ContentType.objects.filter( + Q(app_label="virtualization", model="vminterface") +).first() +interface_ct = ContentType.objects.filter(Q(app_label="dcim", model="interface")).first() - if created: - set_custom_fields_values(ip_address, custom_field_data) +for params in ip_addresses: + custom_field_data = pop_custom_fields(params) - print("🧬 Created IP Address", ip_address.address) + vm = params.pop("virtual_machine", None) + device = params.pop("device", None) + params["address"] = IPNetwork(params["address"]) + + if vm and device: + print("IP Address can only specify one of the following: virtual_machine or device.") + sys.exit() + + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + if assoc == "interface": + if vm: + vm_id = VirtualMachine.objects.get(name=vm).id + query = {"name": params.pop(assoc), "virtual_machine_id": vm_id} + params["assigned_object_type"] = vm_interface_ct + params["assigned_object_id"] = VMInterface.objects.get(**query).id + elif device: + dev_id = Device.objects.get(name=device).id + query = {"name": params.pop(assoc), "device_id": dev_id} + params["assigned_object_type"] = interface_ct + params["assigned_object_id"] = Interface.objects.get(**query).id + else: + query = {field: params.pop(assoc)} + params[assoc] = model.objects.get(**query) + + ip_address, created = IPAddress.objects.get_or_create(**params) + + if created: + set_custom_fields_values(ip_address, custom_field_data) + + print("🧬 Created IP Address", ip_address.address) diff --git a/startup_scripts/270_primary_ips.py b/startup_scripts/270_primary_ips.py index 7a7400e..11c3179 100644 --- a/startup_scripts/270_primary_ips.py +++ b/startup_scripts/270_primary_ips.py @@ -1,44 +1,47 @@ -from dcim.models import Device -from ipam.models import IPAddress -from virtualization.models import VirtualMachine -from startup_script_utils import load_yaml import sys +from dcim.models import Device +from ipam.models import IPAddress +from startup_script_utils import load_yaml +from virtualization.models import VirtualMachine + + def link_primary_ip(assets, asset_model): - for params in assets: - primary_ip_fields = set(params) & {'primary_ip4', 'primary_ip6'} - if not primary_ip_fields: - continue + for params in assets: + primary_ip_fields = set(params) & {"primary_ip4", "primary_ip6"} + if not primary_ip_fields: + continue - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - try: - params[assoc] = model.objects.get(**query) - except model.DoesNotExist: - primary_ip_fields -= {assoc} - print(f"⚠️ IP Address '{query[field]}' not found") + try: + params[assoc] = model.objects.get(**query) + except model.DoesNotExist: + primary_ip_fields -= {assoc} + print(f"⚠️ IP Address '{query[field]}' not found") - asset = asset_model.objects.get(name=params['name']) - for field in primary_ip_fields: - if getattr(asset, field) != params[field]: - setattr(asset, field, params[field]) - print(f"🔗 Define primary IP '{params[field].address}' on '{asset.name}'") - asset.save() + asset = asset_model.objects.get(name=params["name"]) + for field in primary_ip_fields: + if getattr(asset, field) != params[field]: + setattr(asset, field, params[field]) + print(f"🔗 Define primary IP '{params[field].address}' on '{asset.name}'") + asset.save() -devices = load_yaml('/opt/netbox/initializers/devices.yml') -virtual_machines = load_yaml('/opt/netbox/initializers/virtual_machines.yml') + +devices = load_yaml("/opt/netbox/initializers/devices.yml") +virtual_machines = load_yaml("/opt/netbox/initializers/virtual_machines.yml") optional_assocs = { - 'primary_ip4': (IPAddress, 'address'), - 'primary_ip6': (IPAddress, 'address') + "primary_ip4": (IPAddress, "address"), + "primary_ip6": (IPAddress, "address"), } if devices is None and virtual_machines is None: sys.exit() if devices is not None: - link_primary_ip(devices, Device) + link_primary_ip(devices, Device) if virtual_machines is not None: - link_primary_ip(virtual_machines, VirtualMachine) + link_primary_ip(virtual_machines, VirtualMachine) diff --git a/startup_scripts/280_custom_links.py b/startup_scripts/280_custom_links.py index b2a7df5..459ede3 100644 --- a/startup_scripts/280_custom_links.py +++ b/startup_scripts/280_custom_links.py @@ -1,29 +1,33 @@ +import sys + from django.contrib.contenttypes.models import ContentType from extras.models import CustomLink from startup_script_utils import load_yaml -import sys - -custom_links = load_yaml('/opt/netbox/initializers/custom_links.yml') +custom_links = load_yaml("/opt/netbox/initializers/custom_links.yml") if custom_links is None: - sys.exit() + sys.exit() def get_content_type_id(content_type): - try: - return ContentType.objects.get(model=content_type).id - except ContentType.DoesNotExist: - pass + try: + return ContentType.objects.get(model=content_type).id + except ContentType.DoesNotExist: + pass for link in custom_links: - content_type = link.pop('content_type') - link['content_type_id'] = get_content_type_id(content_type) - if link['content_type_id'] is None: - print("⚠️ Unable to create Custom Link '{0}': The content_type '{1}' is unknown".format(link.name, content_type)) - continue + content_type = link.pop("content_type") + link["content_type_id"] = get_content_type_id(content_type) + if link["content_type_id"] is None: + print( + "⚠️ Unable to create Custom Link '{0}': The content_type '{1}' is unknown".format( + link.name, content_type + ) + ) + continue - custom_link, created = CustomLink.objects.get_or_create(**link) - if created: - print("🔗 Created Custom Link '{0}'".format(custom_link.name)) + custom_link, created = CustomLink.objects.get_or_create(**link) + if created: + print("🔗 Created Custom Link '{0}'".format(custom_link.name)) diff --git a/startup_scripts/280_providers.py b/startup_scripts/280_providers.py index e927dda..5c4330a 100644 --- a/startup_scripts/280_providers.py +++ b/startup_scripts/280_providers.py @@ -1,18 +1,19 @@ -from circuits.models import Provider -from startup_script_utils import * import sys -providers = load_yaml('/opt/netbox/initializers/providers.yml') +from circuits.models import Provider +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values + +providers = load_yaml("/opt/netbox/initializers/providers.yml") if providers is None: - sys.exit() + sys.exit() for params in providers: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - provider, created = Provider.objects.get_or_create(**params) + provider, created = Provider.objects.get_or_create(**params) - if created: - set_custom_fields_values(provider, custom_field_data) + if created: + set_custom_fields_values(provider, custom_field_data) - print("📡 Created provider", provider.name) + print("📡 Created provider", provider.name) diff --git a/startup_scripts/290_circuit_types.py b/startup_scripts/290_circuit_types.py index 441f059..071793c 100644 --- a/startup_scripts/290_circuit_types.py +++ b/startup_scripts/290_circuit_types.py @@ -1,18 +1,19 @@ -from circuits.models import CircuitType -from startup_script_utils import * import sys -circuit_types = load_yaml('/opt/netbox/initializers/circuit_types.yml') +from circuits.models import CircuitType +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values + +circuit_types = load_yaml("/opt/netbox/initializers/circuit_types.yml") if circuit_types is None: - sys.exit() + sys.exit() for params in circuit_types: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - circuit_type, created = CircuitType.objects.get_or_create(**params) + circuit_type, created = CircuitType.objects.get_or_create(**params) - if created: - set_custom_fields_values(circuit_type, custom_field_data) + if created: + set_custom_fields_values(circuit_type, custom_field_data) - print("⚡ Created Circuit Type", circuit_type.name) + print("⚡ Created Circuit Type", circuit_type.name) diff --git a/startup_scripts/290_webhooks.py b/startup_scripts/290_webhooks.py index ea8352e..8787f5f 100644 --- a/startup_scripts/290_webhooks.py +++ b/startup_scripts/290_webhooks.py @@ -1,34 +1,34 @@ +import sys + from django.contrib.contenttypes.models import ContentType from extras.models import Webhook from startup_script_utils import load_yaml -import sys - -webhooks = load_yaml('/opt/netbox/initializers/webhooks.yml') +webhooks = load_yaml("/opt/netbox/initializers/webhooks.yml") if webhooks is None: - sys.exit() + sys.exit() def get_content_type_id(hook_name, content_type): - try: - return ContentType.objects.get(model=content_type).id - except ContentType.DoesNotExist as ex: - print("⚠️ Webhook '{0}': The object_type '{1}' is unknown.".format(hook_name, content_type)) - raise ex + try: + return ContentType.objects.get(model=content_type).id + except ContentType.DoesNotExist as ex: + print("⚠️ Webhook '{0}': The object_type '{1}' is unknown.".format(hook_name, content_type)) + raise ex for hook in webhooks: - obj_types = hook.pop('object_types') + obj_types = hook.pop("object_types") - try: - obj_type_ids = [get_content_type_id(hook['name'], obj) for obj in obj_types] - except ContentType.DoesNotExist: - continue + try: + obj_type_ids = [get_content_type_id(hook["name"], obj) for obj in obj_types] + except ContentType.DoesNotExist: + continue - webhook, created = Webhook.objects.get_or_create(**hook) - if created: - webhook.content_types.set(obj_type_ids) - webhook.save() + webhook, created = Webhook.objects.get_or_create(**hook) + if created: + webhook.content_types.set(obj_type_ids) + webhook.save() - print("🪝 Created Webhook {0}".format(webhook.name)) + print("🪝 Created Webhook {0}".format(webhook.name)) diff --git a/startup_scripts/300_circuits.py b/startup_scripts/300_circuits.py index d53272c..f82d3b7 100644 --- a/startup_scripts/300_circuits.py +++ b/startup_scripts/300_circuits.py @@ -1,41 +1,37 @@ -from circuits.models import Circuit, Provider, CircuitType -from tenancy.models import Tenant -from startup_script_utils import * import sys -circuits = load_yaml('/opt/netbox/initializers/circuits.yml') +from circuits.models import Circuit, CircuitType, Provider +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values +from tenancy.models import Tenant + +circuits = load_yaml("/opt/netbox/initializers/circuits.yml") if circuits is None: - sys.exit() + sys.exit() -required_assocs = { - 'provider': (Provider, 'name'), - 'type': (CircuitType, 'name') -} +required_assocs = {"provider": (Provider, "name"), "type": (CircuitType, "name")} -optional_assocs = { - 'tenant': (Tenant, 'name') -} +optional_assocs = {"tenant": (Tenant, "name")} for params in circuits: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - circuit, created = Circuit.objects.get_or_create(**params) + circuit, created = Circuit.objects.get_or_create(**params) - if created: - set_custom_fields_values(circuit, custom_field_data) + if created: + set_custom_fields_values(circuit, custom_field_data) - print("⚡ Created Circuit", circuit.cid) + print("⚡ Created Circuit", circuit.cid) diff --git a/startup_scripts/310_secret_roles.py b/startup_scripts/310_secret_roles.py index 9b0e816..02dcd88 100644 --- a/startup_scripts/310_secret_roles.py +++ b/startup_scripts/310_secret_roles.py @@ -1,14 +1,15 @@ -from secrets.models import SecretRole -from startup_script_utils import load_yaml import sys +from secrets.models import SecretRole -secret_roles = load_yaml('/opt/netbox/initializers/secret_roles.yml') +from startup_script_utils import load_yaml + +secret_roles = load_yaml("/opt/netbox/initializers/secret_roles.yml") if secret_roles is None: - sys.exit() + sys.exit() for params in secret_roles: - secret_role, created = SecretRole.objects.get_or_create(**params) + secret_role, created = SecretRole.objects.get_or_create(**params) - if created: - print("🔑 Created Secret Role", secret_role.name) + if created: + print("🔑 Created Secret Role", secret_role.name) diff --git a/startup_scripts/320_services.py b/startup_scripts/320_services.py index 800f215..a28eb09 100644 --- a/startup_scripts/320_services.py +++ b/startup_scripts/320_services.py @@ -1,29 +1,30 @@ -from ipam.models import Service -from dcim.models import Device -from virtualization.models import VirtualMachine -from startup_script_utils import load_yaml import sys -services = load_yaml('/opt/netbox/initializers/services.yml') +from dcim.models import Device +from ipam.models import Service +from startup_script_utils import load_yaml +from virtualization.models import VirtualMachine + +services = load_yaml("/opt/netbox/initializers/services.yml") if services is None: - sys.exit() + sys.exit() optional_assocs = { - 'device': (Device, 'name'), - 'virtual_machine': (VirtualMachine, 'name') + "device": (Device, "name"), + "virtual_machine": (VirtualMachine, "name"), } for params in services: - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - service, created = Service.objects.get_or_create(**params) + service, created = Service.objects.get_or_create(**params) - if created: - print("🧰 Created Service", service.name) + if created: + print("🧰 Created Service", service.name) diff --git a/startup_scripts/330_power_panels.py b/startup_scripts/330_power_panels.py index 3057a96..bfde18f 100644 --- a/startup_scripts/330_power_panels.py +++ b/startup_scripts/330_power_panels.py @@ -1,41 +1,36 @@ import sys -from dcim.models import Site, RackGroup, PowerPanel -from startup_script_utils import * -from tenancy.models import Tenant +from dcim.models import PowerPanel, RackGroup, Site +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values -power_panels = load_yaml('/opt/netbox/initializers/power_panels.yml') +power_panels = load_yaml("/opt/netbox/initializers/power_panels.yml") if power_panels is None: - sys.exit() + sys.exit() -required_assocs = { - 'site': (Site, 'name') -} +required_assocs = {"site": (Site, "name")} -optional_assocs = { - 'rack_group': (RackGroup, 'name') -} +optional_assocs = {"rack_group": (RackGroup, "name")} for params in power_panels: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - power_panel, created = PowerPanel.objects.get_or_create(**params) + power_panel, created = PowerPanel.objects.get_or_create(**params) - if created: - set_custom_fields_values(power_panel, custom_field_data) + if created: + set_custom_fields_values(power_panel, custom_field_data) - print("⚡ Created Power Panel", power_panel.site, power_panel.name) + print("⚡ Created Power Panel", power_panel.site, power_panel.name) diff --git a/startup_scripts/340_power_feeds.py b/startup_scripts/340_power_feeds.py index fc7ee38..f5aa5b5 100644 --- a/startup_scripts/340_power_feeds.py +++ b/startup_scripts/340_power_feeds.py @@ -1,41 +1,36 @@ import sys -from dcim.models import Rack, PowerPanel, PowerFeed -from startup_script_utils import * -from tenancy.models import Tenant +from dcim.models import PowerFeed, PowerPanel, Rack +from startup_script_utils import load_yaml, pop_custom_fields, set_custom_fields_values -power_feeds = load_yaml('/opt/netbox/initializers/power_feeds.yml') +power_feeds = load_yaml("/opt/netbox/initializers/power_feeds.yml") if power_feeds is None: - sys.exit() + sys.exit() -required_assocs = { - 'power_panel': (PowerPanel, 'name') -} +required_assocs = {"power_panel": (PowerPanel, "name")} -optional_assocs = { - 'rack': (Rack, 'name') -} +optional_assocs = {"rack": (Rack, "name")} for params in power_feeds: - custom_field_data = pop_custom_fields(params) + custom_field_data = pop_custom_fields(params) - for assoc, details in required_assocs.items(): - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in required_assocs.items(): + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - for assoc, details in optional_assocs.items(): - if assoc in params: - model, field = details - query = { field: params.pop(assoc) } + for assoc, details in optional_assocs.items(): + if assoc in params: + model, field = details + query = {field: params.pop(assoc)} - params[assoc] = model.objects.get(**query) + params[assoc] = model.objects.get(**query) - power_feed, created = PowerFeed.objects.get_or_create(**params) + power_feed, created = PowerFeed.objects.get_or_create(**params) - if created: - set_custom_fields_values(power_feed, custom_field_data) + if created: + set_custom_fields_values(power_feed, custom_field_data) - print("⚡ Created Power Feed", power_feed.name) + print("⚡ Created Power Feed", power_feed.name) diff --git a/startup_scripts/__main__.py b/startup_scripts/__main__.py index f85a1de..3a3a3bd 100644 --- a/startup_scripts/__main__.py +++ b/startup_scripts/__main__.py @@ -2,28 +2,30 @@ import runpy from os import scandir -from os.path import dirname, abspath +from os.path import abspath, dirname this_dir = dirname(abspath(__file__)) + def filename(f): - return f.name + return f.name + with scandir(this_dir) as it: - for f in sorted(it, key = filename): - if not f.is_file(): - continue + for f in sorted(it, key=filename): + if not f.is_file(): + continue - if f.name.startswith('__'): - continue + if f.name.startswith("__"): + continue - if not f.name.endswith('.py'): - continue + if not f.name.endswith(".py"): + continue - print(f"▶️ Running the startup script {f.path}") - try: - runpy.run_path(f.path) - except SystemExit as e: - if e.code is not None and e.code != 0: - print(f"‼️ The startup script {f.path} returned with code {e.code}, exiting.") - raise + print(f"▶️ Running the startup script {f.path}") + try: + runpy.run_path(f.path) + except SystemExit as e: + if e.code is not None and e.code != 0: + print(f"‼️ The startup script {f.path} returned with code {e.code}, exiting.") + raise diff --git a/startup_scripts/startup_script_utils/__init__.py b/startup_scripts/startup_script_utils/__init__.py index 1e9f042..2f92370 100644 --- a/startup_scripts/startup_script_utils/__init__.py +++ b/startup_scripts/startup_script_utils/__init__.py @@ -1,3 +1,3 @@ +from .custom_fields import pop_custom_fields, set_custom_fields_values from .load_yaml import load_yaml from .permissions import set_permissions -from .custom_fields import set_custom_fields_values, pop_custom_fields diff --git a/startup_scripts/startup_script_utils/custom_fields.py b/startup_scripts/startup_script_utils/custom_fields.py index 1be7a94..c000970 100644 --- a/startup_scripts/startup_script_utils/custom_fields.py +++ b/startup_scripts/startup_script_utils/custom_fields.py @@ -1,15 +1,16 @@ def set_custom_fields_values(entity, custom_field_data): - if not custom_field_data: - return + if not custom_field_data: + return + + entity.custom_field_data = custom_field_data + return entity.save() - entity.custom_field_data = custom_field_data - return entity.save() def pop_custom_fields(params): - if 'custom_field_data' in params: - return params.pop('custom_field_data') - elif 'custom_fields' in params: - print("⚠️ Please rename 'custom_fields' to 'custom_field_data'!") - return params.pop('custom_fields') + if "custom_field_data" in params: + return params.pop("custom_field_data") + elif "custom_fields" in params: + print("⚠️ Please rename 'custom_fields' to 'custom_field_data'!") + return params.pop("custom_fields") - return None + return None diff --git a/startup_scripts/startup_script_utils/load_yaml.py b/startup_scripts/startup_script_utils/load_yaml.py index ecae7af..048ac5c 100644 --- a/startup_scripts/startup_script_utils/load_yaml.py +++ b/startup_scripts/startup_script_utils/load_yaml.py @@ -1,11 +1,12 @@ from pathlib import Path + from ruamel.yaml import YAML def load_yaml(yaml_file: str): - yf = Path(yaml_file) - if not yf.is_file(): - return None - with yf.open("r") as stream: - yaml = YAML(typ="safe") - return yaml.load(stream) + yf = Path(yaml_file) + if not yf.is_file(): + return None + with yf.open("r") as stream: + yaml = YAML(typ="safe") + return yaml.load(stream) diff --git a/startup_scripts/startup_script_utils/permissions.py b/startup_scripts/startup_script_utils/permissions.py index add83ee..021b0b5 100644 --- a/startup_scripts/startup_script_utils/permissions.py +++ b/startup_scripts/startup_script_utils/permissions.py @@ -2,17 +2,21 @@ from django.contrib.auth.models import Permission def set_permissions(subject, permission_filters): - if subject is None or permission_filters is None: - return - subject.clear() - for permission_filter in permission_filters: - if "*" in permission_filter: - permission_filter_regex = "^" + permission_filter.replace("*", ".*") + "$" - permissions = Permission.objects.filter(codename__iregex=permission_filter_regex) - print(" ⚿ Granting", permissions.count(), "permissions matching '" + permission_filter + "'") - else: - permissions = Permission.objects.filter(codename=permission_filter) - print(" ⚿ Granting permission", permission_filter) + if subject is None or permission_filters is None: + return + subject.clear() + for permission_filter in permission_filters: + if "*" in permission_filter: + permission_filter_regex = "^" + permission_filter.replace("*", ".*") + "$" + permissions = Permission.objects.filter(codename__iregex=permission_filter_regex) + print( + " ⚿ Granting", + permissions.count(), + "permissions matching '" + permission_filter + "'", + ) + else: + permissions = Permission.objects.filter(codename=permission_filter) + print(" ⚿ Granting permission", permission_filter) - for permission in permissions: - subject.add(permission) + for permission in permissions: + subject.add(permission) From 28e4ae44fb77d21c955b063b176610ff8009ee5b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20M=C3=A4der?= Date: Mon, 8 Feb 2021 11:59:57 +0100 Subject: [PATCH 22/28] Lint the syntax of changes --- .ecrc | 23 +++++++++++++++++++++++ .flake8 | 7 +++++++ .github/workflows/push.yml | 22 ++++++++++++++++++++++ .gitignore | 1 + .hadolint.yaml | 3 +++ .markdown-lint.yml | 2 ++ pyproject.toml | 13 +++++++++---- 7 files changed, 67 insertions(+), 4 deletions(-) create mode 100644 .ecrc create mode 100644 .flake8 create mode 100644 .hadolint.yaml create mode 100644 .markdown-lint.yml diff --git a/.ecrc b/.ecrc new file mode 100644 index 0000000..f0c91f7 --- /dev/null +++ b/.ecrc @@ -0,0 +1,23 @@ +{ + "Verbose": false, + "Debug": false, + "IgnoreDefaults": false, + "SpacesAftertabs": false, + "NoColor": false, + "Exclude": [ + "LICENSE", + "\\.initializers", + "\\.vscode" + ], + "AllowedContentTypes": [], + "PassedFiles": [], + "Disable": { + // set these options to true to disable specific checks + "EndOfLine": false, + "Indentation": false, + "InsertFinalNewline": false, + "TrimTrailingWhitespace": false, + "IndentSize": true, + "MaxLineLength": false + } +} diff --git a/.flake8 b/.flake8 new file mode 100644 index 0000000..83a86a2 --- /dev/null +++ b/.flake8 @@ -0,0 +1,7 @@ +[flake8] +max-line-length = 100 +extend-ignore = E203, W503 +per-file-ignores = + configuration/*:E131,E251,E266,E302,E305,E501,E722 + startup_scripts/startup_script_utils/__init__.py:F401 + docker/*:E266,E722 diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml index f7f9a23..bfecd01 100644 --- a/.github/workflows/push.yml +++ b/.github/workflows/push.yml @@ -9,6 +9,28 @@ on: - release jobs: + lint: + runs-on: ubuntu-latest + name: Checks syntax of our code + steps: + - uses: actions/checkout@v2 + - uses: actions/setup-python@v2 + - name: Lint Code Base + uses: github/super-linter@v3 + env: + DEFAULT_BRANCH: develop + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + SUPPRESS_POSSUM: true + LINTER_RULES_PATH: / + VALIDATE_ALL_CODEBASE: false + VALIDATE_DOCKERFILE: false + + EDITORCONFIG_FILE_NAME: .ecrc + DOCKERFILE_HADOLINT_FILE_NAME: .hadolint.yaml + MARKDOWN_CONFIG_FILE: .markdown-lint.yml + PYTHON_BLACK_CONFIG_FILE: pyproject.toml + PYTHON_FLAKE8_CONFIG_FILE: .flake8 + PYTHON_ISORT_CONFIG_FILE: pyproject.toml build: continue-on-error: ${{ matrix.docker_from == 'alpine:edge' }} strategy: diff --git a/.gitignore b/.gitignore index 6eada06..07859bb 100644 --- a/.gitignore +++ b/.gitignore @@ -9,3 +9,4 @@ configuration/* configuration/ldap/* !configuration/ldap/ldap_config.py prometheus.yml +super-linter.log diff --git a/.hadolint.yaml b/.hadolint.yaml new file mode 100644 index 0000000..610e431 --- /dev/null +++ b/.hadolint.yaml @@ -0,0 +1,3 @@ +ignored: +- DL3006 +- DL3018 diff --git a/.markdown-lint.yml b/.markdown-lint.yml new file mode 100644 index 0000000..beebc4b --- /dev/null +++ b/.markdown-lint.yml @@ -0,0 +1,2 @@ +MD013: false +MD041: false diff --git a/pyproject.toml b/pyproject.toml index 3b48bae..cc932eb 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,13 +1,11 @@ [tool.black] -line-length = 100 +line_length = 100 target-version = ['py38'] include = '\.pyi?$' exclude = ''' - ( /( - \.eggs # exclude a few common directories in the - | \.git # root of the project + \.git | \.venv | \.netbox | \.vscode @@ -19,3 +17,10 @@ exclude = ''' [tool.isort] profile = "black" multi_line_output = 3 +line_length = 100 + +[tool.pylint.messages_control] +disable = "C0330, C0326" + +[tool.pylint.format] +max-line-length = "100" From c6df6a040a6e4ef54b65e0907ecb71da890a2ba0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20M=C3=A4der?= Date: Mon, 8 Feb 2021 12:04:22 +0100 Subject: [PATCH 23/28] Fix shellcheck complaint --- build.sh | 2 +- docker/docker-entrypoint.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/build.sh b/build.sh index 408ad0c..47f4160 100755 --- a/build.sh +++ b/build.sh @@ -216,7 +216,7 @@ for DOCKER_TARGET in "${DOCKER_TARGETS[@]}"; do TARGET_DOCKER_TAG="${TARGET_DOCKER_TAG}-${DOCKER_TARGET}" fi if [ -n "${GH_ACTION}" ]; then - echo "FINAL_DOCKER_TAG=${TARGET_DOCKER_TAG}" >> $GITHUB_ENV + echo "FINAL_DOCKER_TAG=${TARGET_DOCKER_TAG}" >> "$GITHUB_ENV" echo "::set-output name=skipped::false" fi diff --git a/docker/docker-entrypoint.sh b/docker/docker-entrypoint.sh index 6d522bf..96ed002 100755 --- a/docker/docker-entrypoint.sh +++ b/docker/docker-entrypoint.sh @@ -8,6 +8,7 @@ set -e umask 002 # Load correct Python3 env +# shellcheck disable=SC1091 source /opt/netbox/venv/bin/activate # Try to connect to the DB From 04ac3d5f4b1aca67495ad19b11aa5e0f78c2fa4f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20M=C3=A4der?= Date: Mon, 8 Feb 2021 12:16:04 +0100 Subject: [PATCH 24/28] Format shell scripts with shfmt --- build-functions/docker-functions.sh | 2 +- build-functions/get-public-image-config.sh | 16 +++--- build-latest.sh | 9 ++-- build.sh | 60 +++++++++++++--------- docker/docker-entrypoint.sh | 8 +-- docker/launch-netbox.sh | 33 ++++++------ test.sh | 4 +- 7 files changed, 73 insertions(+), 59 deletions(-) diff --git a/build-functions/docker-functions.sh b/build-functions/docker-functions.sh index 137ec54..3d947ab 100644 --- a/build-functions/docker-functions.sh +++ b/build-functions/docker-functions.sh @@ -5,4 +5,4 @@ push_image_to_registry() { echo "⏫ Pushing '${target_tag}'" $DRY docker push "${target_tag}" echo "✅ Finished pushing the Docker image '${target_tag}'." -} \ No newline at end of file +} diff --git a/build-functions/get-public-image-config.sh b/build-functions/get-public-image-config.sh index f718716..4f5b695 100644 --- a/build-functions/get-public-image-config.sh +++ b/build-functions/get-public-image-config.sh @@ -45,16 +45,16 @@ _get_image_configuration() { --silent \ --location \ --header "Authorization: Bearer $token" \ - "https://registry-1.docker.io/v2/$image/blobs/$digest" \ - | jq -r ".config.Labels.\"$label\"" + "https://registry-1.docker.io/v2/$image/blobs/$digest" | + jq -r ".config.Labels.\"$label\"" } _get_token() { local image=$1 curl \ --silent \ - "https://auth.docker.io/token?scope=repository:$image:pull&service=registry.docker.io" \ - | jq -r '.token' + "https://auth.docker.io/token?scope=repository:$image:pull&service=registry.docker.io" | + jq -r '.token' } _get_digest() { @@ -65,8 +65,8 @@ _get_digest() { --silent \ --header "Accept: application/vnd.docker.distribution.manifest.v2+json" \ --header "Authorization: Bearer $token" \ - "https://registry-1.docker.io/v2/$image/manifests/$tag" \ - | jq -r '.config.digest' + "https://registry-1.docker.io/v2/$image/manifests/$tag" | + jq -r '.config.digest' } _get_layers() { @@ -77,6 +77,6 @@ _get_layers() { --silent \ --header "Accept: application/vnd.docker.distribution.manifest.v2+json" \ --header "Authorization: Bearer $token" \ - "https://registry-1.docker.io/v2/$image/manifests/$tag" \ - | jq -r '.layers[].digest' + "https://registry-1.docker.io/v2/$image/manifests/$tag" | + jq -r '.layers[].digest' } diff --git a/build-latest.sh b/build-latest.sh index 468dffe..54ebe36 100755 --- a/build-latest.sh +++ b/build-latest.sh @@ -19,7 +19,7 @@ fi # Checking if PRERELEASE is either unset, 'true' or 'false' ### if [ -n "${PRERELEASE}" ] && - { [ "${PRERELEASE}" != "true" ] && [ "${PRERELEASE}" != "false" ]; }; then + { [ "${PRERELEASE}" != "true" ] && [ "${PRERELEASE}" != "false" ]; }; then if [ -z "${DEBUG}" ]; then echo "⚠️ PRERELEASE must be either unset, 'true' or 'false', but was '${PRERELEASE}'!" @@ -60,9 +60,10 @@ if [ "${PRERELEASE}" == "true" ]; then # shellcheck disable=SC2003 MINOR_UNSTABLE=$(expr match "${VERSION}" 'v[0-9]\+\.\([0-9]\+\)') - if { [ "${MAJOR_STABLE}" -eq "${MAJOR_UNSTABLE}" ] \ - && [ "${MINOR_STABLE}" -ge "${MINOR_UNSTABLE}" ]; - } || [ "${MAJOR_STABLE}" -gt "${MAJOR_UNSTABLE}" ]; then + if { + [ "${MAJOR_STABLE}" -eq "${MAJOR_UNSTABLE}" ] && + [ "${MINOR_STABLE}" -ge "${MINOR_UNSTABLE}" ] + } || [ "${MAJOR_STABLE}" -gt "${MAJOR_UNSTABLE}" ]; then echo "❎ Latest unstable version '${VERSION}' is not higher than the latest stable version '$STABLE_VERSION'." if [ -z "$DEBUG" ]; then diff --git a/build.sh b/build.sh index 47f4160..82084bf 100755 --- a/build.sh +++ b/build.sh @@ -117,7 +117,7 @@ NETBOX_PATH="${NETBOX_PATH-.netbox}" ### # Fetching the NetBox source ### -if [ "${2}" != "--push-only" ] && [ -z "${SKIP_GIT}" ] ; then +if [ "${2}" != "--push-only" ] && [ -z "${SKIP_GIT}" ]; then echo "🌐 Checking out '${NETBOX_BRANCH}' of NetBox from the url '${URL}' into '${NETBOX_PATH}'" if [ ! -d "${NETBOX_PATH}" ]; then $DRY git clone -q --depth 10 -b "${NETBOX_BRANCH}" "${URL}" "${NETBOX_PATH}" @@ -174,9 +174,18 @@ PROJECT_VERSION="${PROJECT_VERSION-$(sed -e 's/^[[:space:]]*//' -e 's/[[:space:] # Get the Git information from the netbox directory if [ -d "${NETBOX_PATH}/.git" ]; then - NETBOX_GIT_REF=$(cd "${NETBOX_PATH}"; git rev-parse HEAD) - NETBOX_GIT_BRANCH=$(cd "${NETBOX_PATH}"; git rev-parse --abbrev-ref HEAD) - NETBOX_GIT_URL=$(cd "${NETBOX_PATH}"; git remote get-url origin) + NETBOX_GIT_REF=$( + cd "${NETBOX_PATH}" + git rev-parse HEAD + ) + NETBOX_GIT_BRANCH=$( + cd "${NETBOX_PATH}" + git rev-parse --abbrev-ref HEAD + ) + NETBOX_GIT_URL=$( + cd "${NETBOX_PATH}" + git remote get-url origin + ) fi ### @@ -186,19 +195,22 @@ DOCKER_REGISTRY="${DOCKER_REGISTRY-docker.io}" DOCKER_ORG="${DOCKER_ORG-netboxcommunity}" DOCKER_REPO="${DOCKER_REPO-netbox}" case "${NETBOX_BRANCH}" in - master) - TAG="${TAG-latest}";; - develop) - TAG="${TAG-snapshot}";; - *) - TAG="${TAG-$NETBOX_BRANCH}";; +master) + TAG="${TAG-latest}" + ;; +develop) + TAG="${TAG-snapshot}" + ;; +*) + TAG="${TAG-$NETBOX_BRANCH}" + ;; esac ### # Determine targets to build ### DEFAULT_DOCKER_TARGETS=("main" "ldap") -DOCKER_TARGETS=( "${DOCKER_TARGET:-"${DEFAULT_DOCKER_TARGETS[@]}"}") +DOCKER_TARGETS=("${DOCKER_TARGET:-"${DEFAULT_DOCKER_TARGETS[@]}"}") echo "🏭 Building the following targets:" "${DOCKER_TARGETS[@]}" ### @@ -216,7 +228,7 @@ for DOCKER_TARGET in "${DOCKER_TARGETS[@]}"; do TARGET_DOCKER_TAG="${TARGET_DOCKER_TAG}-${DOCKER_TARGET}" fi if [ -n "${GH_ACTION}" ]; then - echo "FINAL_DOCKER_TAG=${TARGET_DOCKER_TAG}" >> "$GITHUB_ENV" + echo "FINAL_DOCKER_TAG=${TARGET_DOCKER_TAG}" >>"$GITHUB_ENV" echo "::set-output name=skipped::false" fi @@ -242,7 +254,7 @@ for DOCKER_TARGET in "${DOCKER_TARGETS[@]}"; do ### # Proceeding to buils stage, except if `--push-only` is passed ### - if [ "${2}" != "--push-only" ] ; then + if [ "${2}" != "--push-only" ]; then ### # Checking if the build is necessary, # meaning build only if one of those values changed: @@ -259,7 +271,7 @@ for DOCKER_TARGET in "${DOCKER_TARGETS[@]}"; do BUILD_REASON="${BUILD_REASON} interactive" elif [ "$DOCKER_REGISTRY" = "docker.io" ]; then source ./build-functions/get-public-image-config.sh - IFS=':' read -ra DOCKER_FROM_SPLIT <<< "${DOCKER_FROM}" + IFS=':' read -ra DOCKER_FROM_SPLIT <<<"${DOCKER_FROM}" if ! [[ ${DOCKER_FROM_SPLIT[0]} =~ .*/.* ]]; then # Need to use "library/..." for images the have no two part name DOCKER_FROM_SPLIT[0]="library/${DOCKER_FROM_SPLIT[0]}" @@ -295,8 +307,8 @@ for DOCKER_TARGET in "${DOCKER_TARGETS[@]}"; do -t "${TARGET_DOCKER_TAG}" ) if [ -n "${TARGET_DOCKER_SHORT_TAG}" ]; then - DOCKER_BUILD_ARGS+=( -t "${TARGET_DOCKER_SHORT_TAG}" ) - DOCKER_BUILD_ARGS+=( -t "${TARGET_DOCKER_LATEST_TAG}" ) + DOCKER_BUILD_ARGS+=(-t "${TARGET_DOCKER_SHORT_TAG}") + DOCKER_BUILD_ARGS+=(-t "${TARGET_DOCKER_LATEST_TAG}") fi # --label @@ -323,22 +335,22 @@ for DOCKER_TARGET in "${DOCKER_TARGETS[@]}"; do ) fi if [ -n "${BUILD_REASON}" ]; then - BUILD_REASON=$(sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' <<< "$BUILD_REASON") - DOCKER_BUILD_ARGS+=( --label "BUILD_REASON=${BUILD_REASON}" ) + BUILD_REASON=$(sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//' <<<"$BUILD_REASON") + DOCKER_BUILD_ARGS+=(--label "BUILD_REASON=${BUILD_REASON}") fi # --build-arg - DOCKER_BUILD_ARGS+=( --build-arg "NETBOX_PATH=${NETBOX_PATH}" ) + DOCKER_BUILD_ARGS+=(--build-arg "NETBOX_PATH=${NETBOX_PATH}") if [ -n "${DOCKER_FROM}" ]; then - DOCKER_BUILD_ARGS+=( --build-arg "FROM=${DOCKER_FROM}" ) + DOCKER_BUILD_ARGS+=(--build-arg "FROM=${DOCKER_FROM}") fi if [ -n "${HTTP_PROXY}" ]; then - DOCKER_BUILD_ARGS+=( --build-arg "http_proxy=${HTTP_PROXY}" ) - DOCKER_BUILD_ARGS+=( --build-arg "https_proxy=${HTTPS_PROXY}" ) + DOCKER_BUILD_ARGS+=(--build-arg "http_proxy=${HTTP_PROXY}") + DOCKER_BUILD_ARGS+=(--build-arg "https_proxy=${HTTPS_PROXY}") fi if [ -n "${NO_PROXY}" ]; then - DOCKER_BUILD_ARGS+=( --build-arg "no_proxy=${NO_PROXY}" ) + DOCKER_BUILD_ARGS+=(--build-arg "no_proxy=${NO_PROXY}") fi ### @@ -360,7 +372,7 @@ for DOCKER_TARGET in "${DOCKER_TARGETS[@]}"; do ### # Pushing the docker images if either `--push` or `--push-only` are passed ### - if [ "${2}" == "--push" ] || [ "${2}" == "--push-only" ] ; then + if [ "${2}" == "--push" ] || [ "${2}" == "--push-only" ]; then source ./build-functions/docker-functions.sh push_image_to_registry "${TARGET_DOCKER_TAG}" diff --git a/docker/docker-entrypoint.sh b/docker/docker-entrypoint.sh index 96ed002..c795d6a 100755 --- a/docker/docker-entrypoint.sh +++ b/docker/docker-entrypoint.sh @@ -18,7 +18,7 @@ CUR_DB_WAIT_TIME=0 while ! ./manage.py migrate 2>&1 && [ "${CUR_DB_WAIT_TIME}" -lt "${MAX_DB_WAIT_TIME}" ]; do echo "⏳ Waiting on DB... (${CUR_DB_WAIT_TIME}s / ${MAX_DB_WAIT_TIME}s)" sleep "${DB_WAIT_TIMEOUT}" - CUR_DB_WAIT_TIME=$(( CUR_DB_WAIT_TIME + DB_WAIT_TIMEOUT )) + CUR_DB_WAIT_TIME=$((CUR_DB_WAIT_TIME + DB_WAIT_TIMEOUT)) done if [ "${CUR_DB_WAIT_TIME}" -ge "${MAX_DB_WAIT_TIME}" ]; then echo "❌ Waited ${MAX_DB_WAIT_TIME}s or more for the DB to become ready." @@ -36,17 +36,17 @@ else SUPERUSER_EMAIL='admin@example.com' fi if [ -f "/run/secrets/superuser_password" ]; then - SUPERUSER_PASSWORD="$(< /run/secrets/superuser_password)" + SUPERUSER_PASSWORD="$( "../${INITIALIZERS_DIR}/${script}" + sed -E 's/^# //' "${script}" >"../${INITIALIZERS_DIR}/${script}" done ) } From a37f1592fc24cb0770123425accf6272edd8bea1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20M=C3=A4der?= Date: Mon, 8 Feb 2021 18:16:56 +0100 Subject: [PATCH 25/28] Sort env files --- env/netbox.env | 38 +++++++++++++++++++------------------- env/postgres.env | 4 ++-- 2 files changed, 21 insertions(+), 21 deletions(-) diff --git a/env/netbox.env b/env/netbox.env index 8da5259..1f6f896 100644 --- a/env/netbox.env +++ b/env/netbox.env @@ -1,39 +1,39 @@ CORS_ORIGIN_ALLOW_ALL=True -DB_NAME=netbox -DB_USER=netbox -DB_PASSWORD=J5brHrAXFLQSif0K DB_HOST=postgres -EMAIL_SERVER=localhost -EMAIL_PORT=25 -EMAIL_USERNAME=netbox -EMAIL_PASSWORD= -EMAIL_TIMEOUT=5 +DB_NAME=netbox +DB_PASSWORD=J5brHrAXFLQSif0K +DB_USER=netbox EMAIL_FROM=netbox@bar.com +EMAIL_PASSWORD= +EMAIL_PORT=25 +EMAIL_SERVER=localhost +EMAIL_SSL_CERTFILE= +EMAIL_SSL_KEYFILE= +EMAIL_TIMEOUT=5 +EMAIL_USERNAME=netbox # EMAIL_USE_SSL and EMAIL_USE_TLS are mutually exclusive, i.e. they can't both be `true`! EMAIL_USE_SSL=false EMAIL_USE_TLS=false -EMAIL_SSL_CERTFILE= -EMAIL_SSL_KEYFILE= MAX_PAGE_SIZE=1000 MEDIA_ROOT=/opt/netbox/netbox/media METRICS_ENABLED=false -NAPALM_USERNAME= NAPALM_PASSWORD= NAPALM_TIMEOUT=10 -REDIS_HOST=redis -REDIS_PASSWORD=H733Kdjndks81 -REDIS_DATABASE=0 -REDIS_SSL=false +NAPALM_USERNAME= +REDIS_CACHE_DATABASE=1 REDIS_CACHE_HOST=redis-cache REDIS_CACHE_PASSWORD=t4Ph722qJ5QHeQ1qfu36 -REDIS_CACHE_DATABASE=1 REDIS_CACHE_SSL=false +REDIS_DATABASE=0 +REDIS_HOST=redis +REDIS_PASSWORD=H733Kdjndks81 +REDIS_SSL=false RELEASE_CHECK_URL=https://api.github.com/repos/netbox-community/netbox/releases SECRET_KEY=r8OwDznj!!dci#P9ghmRfdu1Ysxm0AiPeDCQhKE+N_rClfWNj SKIP_STARTUP_SCRIPTS=false SKIP_SUPERUSER=false -SUPERUSER_NAME=admin -SUPERUSER_EMAIL=admin@example.com -SUPERUSER_PASSWORD=admin SUPERUSER_API_TOKEN=0123456789abcdef0123456789abcdef01234567 +SUPERUSER_EMAIL=admin@example.com +SUPERUSER_NAME=admin +SUPERUSER_PASSWORD=admin WEBHOOKS_ENABLED=true diff --git a/env/postgres.env b/env/postgres.env index 8bc8d28..bb7b53c 100644 --- a/env/postgres.env +++ b/env/postgres.env @@ -1,3 +1,3 @@ -POSTGRES_USER=netbox -POSTGRES_PASSWORD=J5brHrAXFLQSif0K POSTGRES_DB=netbox +POSTGRES_PASSWORD=J5brHrAXFLQSif0K +POSTGRES_USER=netbox From 4c8435e8744fa9a022382687ed6cc59a6ff5a09a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20M=C3=A4der?= Date: Wed, 10 Feb 2021 10:48:45 +0100 Subject: [PATCH 26/28] fixup! Lint the syntax of changes --- .github/workflows/push.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml index bfecd01..43c6482 100644 --- a/.github/workflows/push.yml +++ b/.github/workflows/push.yml @@ -24,6 +24,7 @@ jobs: LINTER_RULES_PATH: / VALIDATE_ALL_CODEBASE: false VALIDATE_DOCKERFILE: false + FILTER_REGEX_EXCLUDE: (.*/)?(LICENSE|configuration/.*) EDITORCONFIG_FILE_NAME: .ecrc DOCKERFILE_HADOLINT_FILE_NAME: .hadolint.yaml From 8aed79363a88083da0e5775be69508501bbf6c3c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20M=C3=A4der?= Date: Wed, 10 Feb 2021 12:35:06 +0100 Subject: [PATCH 27/28] Move Documentation about Version to Wiki See https://github.com/netbox-community/netbox-docker/wiki/Version. --- README.md | 29 ++--------------------------- 1 file changed, 2 insertions(+), 27 deletions(-) diff --git a/README.md b/README.md index 16c0698..848fece 100644 --- a/README.md +++ b/README.md @@ -97,36 +97,11 @@ If you need help with using NetBox or developing for it or against it's API you This project relies only on *Docker* and *docker-compose* meeting these requirements: -* The *Docker version* must be at least `17.05`. -* The *docker-compose version* must be at least `1.17.0`. +* The *Docker version* must be at least `19.03`. +* The *docker-compose version* must be at least `1.28.0`. To check the version installed on your system run `docker --version` and `docker-compose --version`. -## Use a Specific NetBox Version - -The `docker-compose.yml` file is prepared to run a specific version of NetBox, instead of `latest`. -To use this feature, set and export the environment-variable `VERSION` before launching `docker-compose`, as shown below. -`VERSION` may be set to the name of -[any tag of the `netboxcommunity/netbox` Docker image on Docker Hub][netbox-dockerhub] or [Quay.io][netbox-quayio]. - -```bash -export VERSION=v2.7.1 -docker-compose pull netbox -docker-compose up -d -``` - -You can also build a specific version of the NetBox Docker image yourself. -`VERSION` can be any valid [git ref][git-ref] in that case. - -```bash -export VERSION=v2.7.1 -./build.sh $VERSION -docker-compose up -d -``` - -[git-ref]: https://git-scm.com/book/en/v2/Git-Internals-Git-References -[netbox-github]: https://github.com/netbox-community/netbox/releases - ## Breaking Changes From time to time it might become necessary to re-engineer the structure of this setup. From 74543ec2729ca0ec70990447f65ff00f963c9927 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Christian=20M=C3=A4der?= Date: Wed, 10 Feb 2021 12:47:54 +0100 Subject: [PATCH 28/28] Preparation for 1.0.2 --- VERSION | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VERSION b/VERSION index 7dea76e..6d7de6e 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -1.0.1 +1.0.2