v1.2.0 Updates (#534)

* API client: * Add ip_multi command to support bulk IP Context lookups * Fix issue with "include_invalid" option on quick command failing with "riot" key missing * CLI: * Add ip-multi command to support bulk IP Context lookups * Add support for comma separated IP lists for ip-multi and quick commands * Add size and scroll arguments for query and stats command * Update quick command to not error completely when non-routable IP is passed as an input in a list * Dependencies: * Added colorama dependency * Update Click to 8.0.3 * Updated cachetools to 5.0.0 * Updated jinja to 3.0.3 * Updated more-itertools to 8.12.0 * Updated requests to 2.27.1 * Updated structlog to 21.5.0
GreyNoise-Intelligence · Feb 9, 2022 · 21ef5e4 · 21ef5e4
1 parent d512ace
commit 21ef5e4
Show file tree

Hide file tree

Showing 25 changed files with 641 additions and 84 deletions.
diff --git a/.bumpversion.cfg b/.bumpversion.cfg
@@ -1,5 +1,5 @@
 [bumpversion]
-current_version = 1.1.0
+current_version = 1.2.0
 tag = False
 commit = False
 

diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -0,0 +1,3 @@
+# Github codeowners file to improve our lives when selecting users for PR's
+
+* @bradchiappetta @elliottminns @Obsecurus @superducktoes @nathanqthai
diff --git a/.gitignore b/.gitignore
@@ -90,6 +90,8 @@ venv/
 venv_py2/
 venv3/
 ENV/
+bin/
+pyvenv.cfg
 
 # Spyder project settings
 .spyderproject

diff --git a/CHANGELOG.rst b/CHANGELOG.rst
@@ -2,6 +2,32 @@
 Changelog
 =========
 
+Version `1.2.0`_
+================
+**Date**: September 03, 2021
+
+* API client:
+
+  * Add ip_multi command to support bulk IP Context lookups
+  * Fix issue with "include_invalid" option on quick command failing with "riot" key missing
+
+* CLI:
+
+  * Add ip-multi command to support bulk IP Context lookups
+  * Add support for comma separated IP lists for ip-multi and quick commands
+  * Add size and scroll arguments for query and stats command
+  * Update quick command to not error completely when non-routable IP is passed as an input in a list
+
+* Dependencies:
+
+  * Added colorama dependency
+  * Update Click to 8.0.3
+  * Updated cachetools to 5.0.0
+  * Updated jinja to 3.0.3
+  * Updated more-itertools to 8.12.0
+  * Updated requests to 2.27.1
+  * Updated structlog to 21.5.0
+
 Version `1.1.0`_
 ================
 **Date**: June 23, 2021
@@ -238,3 +264,4 @@ Version `0.2.0`_
 .. _`0.9.1`: https://github.com/GreyNoise-Intelligence/pygreynoise/compare/v0.9.0...0.9.1
 .. _`1.0.0`: https://github.com/GreyNoise-Intelligence/pygreynoise/compare/v0.9.1...1.0.0
 .. _`1.1.0`: https://github.com/GreyNoise-Intelligence/pygreynoise/compare/v1.0.0...1.1.0
+.. _`1.2.0`: https://github.com/GreyNoise-Intelligence/pygreynoise/compare/v1.1.0...1.2.0
diff --git a/docs/source/conf.py b/docs/source/conf.py
@@ -25,7 +25,7 @@
 author = "GreyNoise Intelligence"
 
 # The full version, including alpha/beta/rc tags
-release = "1.1.0"
+release = "1.2.0"
 
 
 # -- General configuration ---------------------------------------------------

diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -12,7 +12,7 @@ Python GreyNoise
 
 License
 =======
-Copyright 2018-2021 GreyNoise Intelligence
+Copyright 2018-2022 GreyNoise Intelligence
 
 Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
 

diff --git a/docs/source/introduction.rst b/docs/source/introduction.rst
@@ -15,4 +15,6 @@ In particular, it will allow you to:
 
 - get a list of noise IP address found in a given date.
 
-.. _GreyNoise API: https://developer.greynoise.io/reference
+- check if an IP address belongs to a common business service
+
+.. _GreyNoise API: https://docs.greynoise.io/reference
diff --git a/docs/source/tutorial.rst b/docs/source/tutorial.rst
@@ -136,21 +136,89 @@ Detailed context information for any given IP address is also available::
       }
     }
 
+When there's a list of IP addresses to get full context from, they can be checked all at once like
+this (this method also supports the include_invalid flag::
+
+    >>> api_client.ip_multi(['8.8.8.8', '58.220.219.247'])
+      [
+    {
+      'ip': '8.8.8.8',
+      'first_seen': '',
+      'last_seen': '',
+      'seen': False,
+      'tags': None,
+      'actor': '',
+      'spoofable': False,
+      'classification': '',
+      'cve': None,
+      'bot': False,
+      'vpn': False,
+      'vpn_service': '',
+      'metadata': {
+        'asn': '',
+        'city': '',
+        'country': '',
+        'country_code': '',
+        'organization': '',
+        'category': '',
+        'tor': False,
+        'rdns': '',
+        'os': ''
+      },
+      'raw_data': {
+        'scan': [],
+        'web': {},
+        'ja3': [],
+        'hassh': []
+      }
+    },
+    {
+      'ip': '58.220.219.247',
+      'first_seen': '',
+      'last_seen': '',
+      'seen': False,
+      'tags': None,
+      'actor': '',
+      'spoofable': False,
+      'classification': '',
+      'cve': None,
+      'bot': False,
+      'vpn': False,
+      'vpn_service': '',
+      'metadata': {
+        'asn': '',
+        'city': '',
+        'country': '',
+        'country_code': '',
+        'organization': '',
+        'category': '',
+        'tor': False,
+        'rdns': '',
+        'os': ''
+      },
+      'raw_data': {
+        'scan': [],
+        'web': {},
+        'ja3': [],
+        'hassh': []
+      }
+    }
+  ]
+
 Any IP can also be checked to see if it exists within the RIOT dataset::
 
-    >>> api_client.riot('58.220.219.247')
+    >>> api_client.riot('8.8.8.8')
     {
-        'ip': '8.8.8.8',
-        'riot': True,
-        'category': 'public_dns',
-        'name': 'Google Public DNS',
-        'description': "Google's global domain name system (DNS) resolution service.",
-        'explanation': "Public DNS services are used as alternatives to ISP's name servers. You may
-        see devices on your network communicating with Google Public DNS over port 53/TCP or 53/UDP
-        to resolve DNS lookups.",
-        'last_updated': '2021-01-06T01:56:45Z',
-        'logo_url': 'https://www.gstatic.com/devrel-devsite/prod/v9d82702993bc22f782b7874a0f933b5e39c1f0889acab7d1fce0d6deb8e0f63d/cloud/images/cloud-logo.svg',
-        'reference': 'https://developers.google.com/speed/public-dns/docs/isp#alternative'
+      'ip': '8.8.8.8',
+      'riot': True,
+      'category': 'public_dns',
+      'name': 'Google Public DNS',
+      'description': "Google's global domain name system (DNS) resolution service.",
+      'explanation': "Public DNS services are used as alternatives to ISP's name servers. You may see devices on your network communicating with Google Public DNS over port 53/TCP or 53/UDP to resolve DNS lookups.",
+      'last_updated': '2022-02-08T18:58:27Z',
+      'logo_url': 'https://upload.wikimedia.org/wikipedia/commons/2/2f/Google_2015_logo.svg',
+      'reference': 'https://developers.google.com/speed/public-dns/docs/isp#alternative',
+      'trust_level': '1'
     }
 
 .. note::
@@ -505,7 +573,7 @@ Internet as follows::
    58.220.219.247 is classified as NOISE.
 
 When there's a list of IP addresses to verify, they can be checked all at once like
-this::
+this (a comma seperated list is also supported::
 
    $ greynoise quick 8.8.8.8 58.220.219.247
    8.8.8.8 is classified as NOT NOISE.
@@ -548,6 +616,44 @@ Detailed context information for any given IP address is also available::
    - Port/Proto: 3389/TCP
    - Port/Proto: 65529/TCP
 
+When there's a list of IP addresses to verify, they can be checked all at once like
+this (a comma seperated list is also supported::
+
+   $ greynoise ip-multi 8.8.8.8 58.220.219.247
+          OVERVIEW
+    ----------------------------
+    Actor: unknown
+    Classification: malicious
+    First seen: 2020-12-21
+    IP: 42.230.170.174
+    Last seen: 2022-02-08
+    Tags:
+    - Mirai
+
+              METADATA
+    ----------------------------
+    ASN: AS4837
+    Category: isp
+    Location:
+    Region: Heilongjiang
+    Organization: CHINA UNICOM China169 Backbone
+    OS: Linux 2.2-3.x
+    rDNS: hn.kd.ny.adsl
+    Spoofable: False
+    Tor: False
+
+              RAW DATA
+    ----------------------------
+    [Scan]
+    - Port/Proto: 23/TCP
+    - Port/Proto: 8080/TCP
+
+    [Paths]
+    - /setup.cgi
+
+    8.8.8.8 is classified as NOT NOISE.
+
+
 
 GNQL
 ----

diff --git a/requirements/common.txt b/requirements/common.txt
@@ -1,14 +1,15 @@
-Click==8.0.1
+Click==8.0.3
 ansimarkup==1.4.0
-cachetools==4.2.2;python_version>='3'
+cachetools==5.0.0;python_version>='3'
+colorama==0.4.4
 click-default-group==1.2.2
 click-repl==0.2.0
 dict2xml==1.7.0;python_version>='3'
 ipaddress==1.0.23
 jinja2==2.11.3;python_version=='3.5' # pyup: ignore
-jinja2==3.0.1;python_version>='3.6'
-more-itertools==8.8.0;python_version>='3'
-requests==2.25.1
+jinja2==3.0.3;python_version>='3.6'
+more-itertools==8.12.0;python_version>='3'
+requests==2.27.1
 six==1.16.0
 structlog==20.1.0;python_version=='3.5' # pyup: ignore
-structlog==21.1.0;python_version>='3.6'
+structlog==21.5.0;python_version>='3.6'
diff --git a/requirements/dev.txt b/requirements/dev.txt
@@ -1,6 +1,6 @@
 # Requirements needed to develop the application
 -r test.txt
 advbumpversion==1.2.0
-ipython==7.25.0;python_version>='3'
-pre-commit==2.13.0
-tox==3.23.1
+ipython==8.0.1;python_version>='3'
+pre-commit==2.17.0
+tox==3.24.5
diff --git a/requirements/docs.txt b/requirements/docs.txt
@@ -1,5 +1,5 @@
 # Requirements needed to build the documentation
 -r common.txt
-Sphinx==4.0.2
-sphinx-click==3.0.1
-sphinx-rtd-theme==0.5.2
+Sphinx==4.4.0
+sphinx-click==3.0.3
+sphinx-rtd-theme==1.0.0
diff --git a/requirements/test.txt b/requirements/test.txt
@@ -1,14 +1,16 @@
-black==21.6b0;python_version>='3.6'
-flake8==3.9.2
+black==22.1.0;python_version>='3.6'
+flake8==3.9.2;python_version=='3.5' # pyup: ignore
+flake8==4.0.1;python_version>='3.6'
 isort==4.3.21;python_version=='3.5' # pyup: ignore
-isort==5.9.1;python_version>='3.6'
+isort==5.10.1;python_version>='3.6'
 mock==3.0.5;python_version=='3.5' # pyup: ignore
 mock==4.0.3;python_version>='3.6'
 pylint==2.6.2;python_version=='3.5' # pyup: ignore
-pylint==2.9.0;python_version>='3.6'
-pytest-cov==2.12.1
+pylint==2.12.2;python_version>='3.6'
+pytest-cov==2.12.1;python_version=='3.5' # pyup: ignore
+pytest-cov==3.0.0;python_version>='3.6'
 pytest==6.1.2;python_version=='3.5' # pyup: ignore
-pytest==6.2.4;python_version>='3.6'
+pytest==6.2.5;python_version>='3.6'
 restructuredtext-lint==1.3.2
-twine==3.4.1;python_version>='3.6'
-yamllint==1.26.1;python_version>='3'
+twine==3.7.1;python_version>='3.6'
+yamllint==1.26.3;python_version>='3'
diff --git a/setup.cfg b/setup.cfg
@@ -1,6 +1,7 @@
 [flake8]
 max-line-length = 88
 max-complexity = 10
+ignore = C901,W503
 
 [isort]
 multi_line_output = 3

diff --git a/setup.py b/setup.py
@@ -27,7 +27,7 @@ def read(fname):
 
 setup(
     name="greynoise",
-    version="1.1.0",
+    version="1.2.0",
     description="Abstraction to interact with GreyNoise API.",
     url="https://greynoise.io/",
     author="GreyNoise Intelligence",

diff --git a/src/greynoise/__version__.py b/src/greynoise/__version__.py
@@ -5,4 +5,4 @@
 __maintainer__ = "GreyNoise Intelligence"
 __email__ = "[email protected]"
 __status__ = "BETA"
-__version__ = "1.1.0"
+__version__ = "1.2.0"
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,3 @@
		# Github codeowners file to improve our lives when selecting users for PR's

		* @bradchiappetta @elliottminns @Obsecurus @superducktoes @nathanqthai
-Original file line number
+Diff line change
@@ Expand Up / @@ -12,7 +12,7 @@ Python GreyNoise @@
     License
     =======
-    Copyright 2018-2021 GreyNoise Intelligence
+    Copyright 2018-2022 GreyNoise Intelligence
     Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
@@ Expand Down @@