# msticnb - Notebooklets for Jupyter Notebooks
Read the full documentation at [msticnb.readthedocs](https://msticnb.readthedocs.io/en/latest/)
msticnb is a companion package to
[msticpy](https://msticpy.readthedocs.io/en/latest/). It is designed to
be used in Jupyter notebooks by security operations engineers and analysts,
to give them quick access to
common notebook patterns such as retrieving summary information about
a host or IP address.
<img src="https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-browser.png"
alt="Notebooklet browser showing list of notebooklets and some details of the user documentation for the selected notebooklet."
title="Notebooklet browser"
height="300" />
Each notebooklet is equivalent to multiple cells and many lines of code
in a traditional notebook. You can import and run a notebooklet with two
lines of code (or even 1 line, if you are impatient). Typically, the input
parameters to a notebooklet will be an identifier (e.g. a host name) and
a time range (over which to query data). Some notebooklets (primarily
packaged analytics) will take a pandas DataFrame as input.
```python
host_summary = nb.nblts.azsent.host.HostSummary()
host_sum_rslt = host_summary.run(
value="Msticalertswin1", timespan=time_span
)
```
You can create your own notebooklets and use them in the same framework
as the ones already in the package.
---
## Notebooklets
### What are notebooklets?
Notebooklets are collections of notebook cells that implement some
useful reusable sequence. They are extensions of, and build upon the
msticpy package and are design to streamline authoring of Jupyter
notebooks for CyberSec hunters and investigators. The goal of
notebooklets is to replace repetitive and lengthy boilerplate code in
notebooks for common operations.
Some examples are:
- Get a host summary for a named host (IP address, cloud registration
information, recent alerts)
- Get account activity for an account (host and cloud logons and
failures, summary of recent activity)
- Triage alerts with Threat Intel data (prioritize your alerts by
correlating with Threat intel sources)
### Intended Audience
- Cyber security investigators and hunters using Jupyter notebooks for
their work
- Security Ops Center (SOC) engineers/SecDevOps building reusable
notebooks for SOC analysts
### Why did we create notebooklets?
- Notebook code can quickly become complex and lengthy:
- obscures the information you are trying to display
- can be intimidating to non-developers
- Code in notebook code cells is not easily re-useable:
- You can copy and paste but how do you sync changes back to the
original notebook?
- Difficult to discover code snippets in notebooks
- Notebook code is often fragile:
- Often not parameterized or modular
- Code blocks are frequently dependent on global values assigned
earlier
- Output data is not in any standard format
- Difficult to test
### Why aren\'t these part of msticpy?
- Msticpy aims to be platform-independent, whereas most if not all
notebooklets assume a data schema that is specific to their data
provider/SIEM.
- Msticpy is mostly for discrete functions such as data acquisition,
analysis and visualization. Msticnb implements common SOC scenarios
using this functionality.
### Traditional Notebook vs. one using a Notebooklets
The notebook on the left is using mostly inline code (occupying more
than 50% of the notebook). The one on the right is using a single
notebooklet with only 3 or 4 lines of code.
<img src="https://github.com/microsoft/msticnb/blob/master/docs/source/_static/NBComparison.png"
alt="Comparing a standard notebook with one using a notebooklet. The standard notebook on the left can require large amounts of code. The notebook on the right uses just 3 lines of code."
title="With and without notebooklets" height="500" />
### Characteristics of Notebooklets
- They have one or small number of entry points/methods (typically a
\"run\" method)
- They are parametrizable (e.g. you can supply hostname, IP Address,
time range, etc.) and they may have runtime options to allow to skip
unwanted processing or include optional processing
- They can query, process or visualize data (or any combination)
- They return a package of results that can be used later in the
notebook
- The code can be imported into a notebook cell to be modified, if
needed.
### Limitations
- They are normally specific to a data backend/SIEM since the data
schema and layout varies between SIEM vendors.
- Notebooklet code layout is typically more complex than standard
notebook code
---
## Using Notebooklets
For a more detailed explanation of these steps and illustration of other
features see the [Notebooklets
notebook](https://github.com/microsoft/msticnb/blob/master/docs/notebooks/NotebookletsDemo.ipynb)
### Install the Package
```bash
pip install msticnb
```
### Import and initialize
<img src="https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-import.png"
alt="Python statement to import msticnb - 'import msticnb as nb'"
title="Importing" height="70" />
The init method loads data drivers and data providers relevant to the
the chosen data platform.
<img src="https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-init.png"
alt="Python statement to initialize msticnb - nb.init('AzureSentinel')"
title="Initializing msticnb" height="70" />
### Pick a notebooklet to use
You can pick a notebooklet from the commandline, using autocompletion.
You can also search for a notebooklet using keywords and text from the
notebooklet name and documentation.
The easiest way is using the nb.browse() method. This lists all of the
available notebooklets and displays documentation, usage information and
sample code snippet for each.
<img src="https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-browser.png"
alt="Notebooklet browser showing list of notebooklets and some details of the user documentation for the selected notebooklet."
title="Notebooklet browser" height="500" />
### Instantiate the notebooklet and execute \"run\"
Notebooklets usually have a single `run` method, which is the entry
point for the notebooklet. A notebooklet might have additional methods
to do further drill-down, data retrieval, visualization or other
operations once the run method has completed. Run typically requires
parameters such as a host or account identifier and a time range over
which to perform the operations.
<img src="https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-run-cell.png"
alt="Python code cell showing the creation of a notebooklet instance from the WinHostevents notebooklet class. The notebooklet 'run' method is called with parameters supplying the name of the host and a time range."
title="Running a notebooklet" height="100" />
The notebooklet displays output directly to the notebook (although this
can be suppressed) - showing text, data tables and visualizations. This
data is all saved to a Results object. The data items are simple
properties of this results object, for example, DataFrames, plots, or
simple Python dictionaries. You can access these individually and you
can just display the results object using IPython display() or just
typing its name into and emtpy cell and running the cell.
<img src="https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-run.png"
alt="The notebooklet displays output directly to th notebook. The output includes styled tables, text headings and descriptions and interactive timeline visualizations."
title="Running a notebooklet" height="600" />
### View extended help for a notebooklet
You can access detailed documentation from any notebooklet class or
instance using the show_help() method. This help includes a high-level
description and usage information (parameters, available methods,
options). It also describes the major output sections that will be
displayed and the the contents of the return results.
Note: the contents of this help are also displayed in the notebooklet browser
shown earlier.
<img src="https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-help.png"
alt="The notebooklet help displays a description, parameter and other usage information and available methods. It also describes the major output sections and the contents of the return results."
title="Notebooklet help" height="500" />
## Current Notebooklets
### AccountSummary
Retrieves account summary for the selected account.
Main operations:
- Searches for matches for the account name in Active Directory,
Windows and Linux host logs.
- If one or more matches are found it will return a selection widget
that you can use to pick the account.
- Selecting the account displays a summary of recent activity and
retrieves any alerts and hunting bookmarks related to the account
- The alerts and bookmarks are browsable using the browse_alerts and
browse_bookmarks methods
- You can call the find_additional_data method to retrieve and display
more detailed activity information for the account (e.g. host logons,
Azure and Office 365 activity)
### EnrichAlerts
Alert Enrichment Notebooklet Class.
Enriches Azure Sentinel alerts with Threat Intelligence and other data.
### HostLogonsSummary
Host Logons Summary Notebooklet class.
Queries and displays information about logons to a host including:
- Summary of successful logons
- Visualizations of logon event times
- Geolocation of remote logon sources
- Visualizations of various logon elements depending on host type
- Data on users with failed and successful logons
### HostSummary
HostSummary Notebooklet class.
Queries and displays information about a host including:
- IP address assignment
- Related alerts
- Related hunting/investigation bookmarks
- Azure subscription/resource data.
### WinHostEvents
Windows host Security Events Notebooklet class.
Queries and displays Windows Security Events including:
- All security events summary
- Extracting and displaying account management events
- Account management event timeline
- Optionally parsing packed event data into DataFrame columns
Process (4688) and Account Logon (4624, 4625) are not included in the
event types processed by this module.
### IpAddressSummary
Retrieves common data about an IP Address including:
- Tries to determine IP address is external or internal (i.e. owned by the organization)
- Azure Heartbeat, Network Analytics or VMComputer records
- Geo-IP and Whois data
- Threat intel reports
- Related alerts and hunting bookmarks
- Network flows involving IP address
- Azure activity (e.g. sign-ins) originating from IP address
### NetworkFlowSummary
Network Flow Summary Notebooklet class.
Queries network data and plots time lines for network traffic to/from a
host or IP address.
- Plot flows events by protocol and direction
- Plot flow count by protocol
- Display flow summary table
- Display flow summary by ASN
- Display results on map
### TemplateNB
Template Notebooklet class.
A code template for creating additional notebooklets.
## See Also
[msticpy documentation](https://msticpy.readthedocs.io/en/latest/)
Raw data
{
"_id": null,
"home_page": "https://github.com/microsoft/msticnb",
"name": "msticnb",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": null,
"keywords": "security, cybersecurity, infosec, jupyter, notebook, azure, sentinel",
"author": "Ian Hellen",
"author_email": "ianhelle@microsoft.com",
"download_url": "https://files.pythonhosted.org/packages/df/68/14d700b1717053184435c7548ef3a53c3caf4cfb8bb7bb037cfe5c4a9cf3/msticnb-1.2.3.tar.gz",
"platform": null,
"description": "# msticnb - Notebooklets for Jupyter Notebooks\n\nRead the full documentation at [msticnb.readthedocs](https://msticnb.readthedocs.io/en/latest/)\n\nmsticnb is a companion package to\n[msticpy](https://msticpy.readthedocs.io/en/latest/). It is designed to\nbe used in Jupyter notebooks by security operations engineers and analysts,\nto give them quick access to\ncommon notebook patterns such as retrieving summary information about\na host or IP address.\n\n<img src=\"https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-browser.png\"\nalt=\"Notebooklet browser showing list of notebooklets and some details of the user documentation for the selected notebooklet.\"\ntitle=\"Notebooklet browser\"\nheight=\"300\" />\n\nEach notebooklet is equivalent to multiple cells and many lines of code\nin a traditional notebook. You can import and run a notebooklet with two\nlines of code (or even 1 line, if you are impatient). Typically, the input\nparameters to a notebooklet will be an identifier (e.g. a host name) and\na time range (over which to query data). Some notebooklets (primarily\npackaged analytics) will take a pandas DataFrame as input.\n\n```python\n host_summary = nb.nblts.azsent.host.HostSummary()\n host_sum_rslt = host_summary.run(\n value=\"Msticalertswin1\", timespan=time_span\n )\n```\n\nYou can create your own notebooklets and use them in the same framework\nas the ones already in the package.\n\n---\n\n## Notebooklets\n\n### What are notebooklets?\n\nNotebooklets are collections of notebook cells that implement some\nuseful reusable sequence. They are extensions of, and build upon the\nmsticpy package and are design to streamline authoring of Jupyter\nnotebooks for CyberSec hunters and investigators. The goal of\nnotebooklets is to replace repetitive and lengthy boilerplate code in\nnotebooks for common operations.\n\nSome examples are:\n\n- Get a host summary for a named host (IP address, cloud registration\n information, recent alerts)\n- Get account activity for an account (host and cloud logons and\n failures, summary of recent activity)\n- Triage alerts with Threat Intel data (prioritize your alerts by\n correlating with Threat intel sources)\n\n### Intended Audience\n\n- Cyber security investigators and hunters using Jupyter notebooks for\n their work\n- Security Ops Center (SOC) engineers/SecDevOps building reusable\n notebooks for SOC analysts\n\n### Why did we create notebooklets?\n\n- Notebook code can quickly become complex and lengthy:\n - obscures the information you are trying to display\n - can be intimidating to non-developers\n- Code in notebook code cells is not easily re-useable:\n - You can copy and paste but how do you sync changes back to the\n original notebook?\n - Difficult to discover code snippets in notebooks\n- Notebook code is often fragile:\n - Often not parameterized or modular\n - Code blocks are frequently dependent on global values assigned\n earlier\n - Output data is not in any standard format\n - Difficult to test\n\n### Why aren\\'t these part of msticpy?\n\n- Msticpy aims to be platform-independent, whereas most if not all\n notebooklets assume a data schema that is specific to their data\n provider/SIEM.\n- Msticpy is mostly for discrete functions such as data acquisition,\n analysis and visualization. Msticnb implements common SOC scenarios\n using this functionality.\n\n### Traditional Notebook vs. one using a Notebooklets\n\nThe notebook on the left is using mostly inline code (occupying more\nthan 50% of the notebook). The one on the right is using a single\nnotebooklet with only 3 or 4 lines of code.\n\n<img src=\"https://github.com/microsoft/msticnb/blob/master/docs/source/_static/NBComparison.png\"\nalt=\"Comparing a standard notebook with one using a notebooklet. The standard notebook on the left can require large amounts of code. The notebook on the right uses just 3 lines of code.\"\ntitle=\"With and without notebooklets\" height=\"500\" />\n\n\n### Characteristics of Notebooklets\n\n- They have one or small number of entry points/methods (typically a\n \\\"run\\\" method)\n- They are parametrizable (e.g. you can supply hostname, IP Address,\n time range, etc.) and they may have runtime options to allow to skip\n unwanted processing or include optional processing\n- They can query, process or visualize data (or any combination)\n- They return a package of results that can be used later in the\n notebook\n- The code can be imported into a notebook cell to be modified, if\n needed.\n\n### Limitations\n\n- They are normally specific to a data backend/SIEM since the data\n schema and layout varies between SIEM vendors.\n- Notebooklet code layout is typically more complex than standard\n notebook code\n\n---\n\n## Using Notebooklets\n\nFor a more detailed explanation of these steps and illustration of other\nfeatures see the [Notebooklets\nnotebook](https://github.com/microsoft/msticnb/blob/master/docs/notebooks/NotebookletsDemo.ipynb)\n\n### Install the Package\n\n```bash\npip install msticnb\n```\n\n### Import and initialize\n\n<img src=\"https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-import.png\"\nalt=\"Python statement to import msticnb - 'import msticnb as nb'\"\ntitle=\"Importing\" height=\"70\" />\n\nThe init method loads data drivers and data providers relevant to the\nthe chosen data platform.\n\n<img src=\"https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-init.png\"\nalt=\"Python statement to initialize msticnb - nb.init('AzureSentinel')\"\ntitle=\"Initializing msticnb\" height=\"70\" />\n\n### Pick a notebooklet to use\n\nYou can pick a notebooklet from the commandline, using autocompletion.\nYou can also search for a notebooklet using keywords and text from the\nnotebooklet name and documentation.\n\nThe easiest way is using the nb.browse() method. This lists all of the\navailable notebooklets and displays documentation, usage information and\nsample code snippet for each.\n\n<img src=\"https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-browser.png\"\nalt=\"Notebooklet browser showing list of notebooklets and some details of the user documentation for the selected notebooklet.\"\ntitle=\"Notebooklet browser\" height=\"500\" />\n\n### Instantiate the notebooklet and execute \\\"run\\\"\n\nNotebooklets usually have a single `run` method, which is the entry\npoint for the notebooklet. A notebooklet might have additional methods\nto do further drill-down, data retrieval, visualization or other\noperations once the run method has completed. Run typically requires\nparameters such as a host or account identifier and a time range over\nwhich to perform the operations.\n\n<img src=\"https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-run-cell.png\"\nalt=\"Python code cell showing the creation of a notebooklet instance from the WinHostevents notebooklet class. The notebooklet 'run' method is called with parameters supplying the name of the host and a time range.\"\ntitle=\"Running a notebooklet\" height=\"100\" />\n\nThe notebooklet displays output directly to the notebook (although this\ncan be suppressed) - showing text, data tables and visualizations. This\ndata is all saved to a Results object. The data items are simple\nproperties of this results object, for example, DataFrames, plots, or\nsimple Python dictionaries. You can access these individually and you\ncan just display the results object using IPython display() or just\ntyping its name into and emtpy cell and running the cell.\n\n<img src=\"https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-run.png\"\nalt=\"The notebooklet displays output directly to th notebook. The output includes styled tables, text headings and descriptions and interactive timeline visualizations.\"\ntitle=\"Running a notebooklet\" height=\"600\" />\n\n### View extended help for a notebooklet\n\nYou can access detailed documentation from any notebooklet class or\ninstance using the show_help() method. This help includes a high-level\ndescription and usage information (parameters, available methods,\noptions). It also describes the major output sections that will be\ndisplayed and the the contents of the return results.\n\nNote: the contents of this help are also displayed in the notebooklet browser\nshown earlier.\n\n<img src=\"https://github.com/microsoft/msticnb/blob/master/docs/source/_static/msticnb-help.png\"\nalt=\"The notebooklet help displays a description, parameter and other usage information and available methods. It also describes the major output sections and the contents of the return results.\"\ntitle=\"Notebooklet help\" height=\"500\" />\n\n## Current Notebooklets\n\n### AccountSummary\n\nRetrieves account summary for the selected account.\n\nMain operations:\n\n- Searches for matches for the account name in Active Directory,\n Windows and Linux host logs.\n- If one or more matches are found it will return a selection widget\n that you can use to pick the account.\n- Selecting the account displays a summary of recent activity and\n retrieves any alerts and hunting bookmarks related to the account\n- The alerts and bookmarks are browsable using the browse_alerts and\n browse_bookmarks methods\n- You can call the find_additional_data method to retrieve and display\n more detailed activity information for the account (e.g. host logons,\n Azure and Office 365 activity)\n\n### EnrichAlerts\n\nAlert Enrichment Notebooklet Class.\n\nEnriches Azure Sentinel alerts with Threat Intelligence and other data.\n\n### HostLogonsSummary\n\nHost Logons Summary Notebooklet class.\n\nQueries and displays information about logons to a host including:\n\n- Summary of successful logons\n- Visualizations of logon event times\n- Geolocation of remote logon sources\n- Visualizations of various logon elements depending on host type\n- Data on users with failed and successful logons\n\n### HostSummary\n\nHostSummary Notebooklet class.\n\nQueries and displays information about a host including:\n\n- IP address assignment\n- Related alerts\n- Related hunting/investigation bookmarks\n- Azure subscription/resource data.\n\n### WinHostEvents\n\nWindows host Security Events Notebooklet class.\n\nQueries and displays Windows Security Events including:\n\n- All security events summary\n- Extracting and displaying account management events\n- Account management event timeline\n- Optionally parsing packed event data into DataFrame columns\n\nProcess (4688) and Account Logon (4624, 4625) are not included in the\nevent types processed by this module.\n\n### IpAddressSummary\n\nRetrieves common data about an IP Address including:\n\n- Tries to determine IP address is external or internal (i.e. owned by the organization)\n- Azure Heartbeat, Network Analytics or VMComputer records\n- Geo-IP and Whois data\n- Threat intel reports\n- Related alerts and hunting bookmarks\n- Network flows involving IP address\n- Azure activity (e.g. sign-ins) originating from IP address\n\n### NetworkFlowSummary\n\nNetwork Flow Summary Notebooklet class.\n\nQueries network data and plots time lines for network traffic to/from a\nhost or IP address.\n\n- Plot flows events by protocol and direction\n- Plot flow count by protocol\n- Display flow summary table\n- Display flow summary by ASN\n- Display results on map\n\n### TemplateNB\n\nTemplate Notebooklet class.\n\nA code template for creating additional notebooklets.\n\n## See Also\n\n[msticpy documentation](https://msticpy.readthedocs.io/en/latest/)\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "MSTIC Notebooklets",
"version": "1.2.3",
"project_urls": {
"Code": "https://github.com/microsoft/msticnb",
"Documentation": "https://msticnb.readthedocs.io",
"Homepage": "https://github.com/microsoft/msticnb"
},
"split_keywords": [
"security",
" cybersecurity",
" infosec",
" jupyter",
" notebook",
" azure",
" sentinel"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f81a330d02863f896811c0379e0b4b1c410582e64c021061929d287a82653b8e",
"md5": "6ab93b8c74ca2433cc138b1b70acc3a0",
"sha256": "cd8182760ebc47674c3575a603e8ee1d8e9484e5a530ae453d05cae5cd14da23"
},
"downloads": -1,
"filename": "msticnb-1.2.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "6ab93b8c74ca2433cc138b1b70acc3a0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 121605,
"upload_time": "2024-09-24T03:02:55",
"upload_time_iso_8601": "2024-09-24T03:02:55.464101Z",
"url": "https://files.pythonhosted.org/packages/f8/1a/330d02863f896811c0379e0b4b1c410582e64c021061929d287a82653b8e/msticnb-1.2.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "df6814d700b1717053184435c7548ef3a53c3caf4cfb8bb7bb037cfe5c4a9cf3",
"md5": "5b329cd04ac95321054350c56fde5ffc",
"sha256": "372c95cc3333e9bc1ea0a72301210355908b4b584bc0d342f9e2514e190f5a72"
},
"downloads": -1,
"filename": "msticnb-1.2.3.tar.gz",
"has_sig": false,
"md5_digest": "5b329cd04ac95321054350c56fde5ffc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 104174,
"upload_time": "2024-09-24T03:02:57",
"upload_time_iso_8601": "2024-09-24T03:02:57.026456Z",
"url": "https://files.pythonhosted.org/packages/df/68/14d700b1717053184435c7548ef3a53c3caf4cfb8bb7bb037cfe5c4a9cf3/msticnb-1.2.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-24 03:02:57",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "microsoft",
"github_project": "msticnb",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "bokeh",
"specs": [
[
"<",
"3.4.0"
],
[
">=",
"1.4.0"
]
]
},
{
"name": "defusedxml",
"specs": [
[
">=",
"0.6.0"
]
]
},
{
"name": "ipython",
"specs": [
[
">=",
"7.23.1"
]
]
},
{
"name": "ipywidgets",
"specs": [
[
">=",
"7.5.1"
]
]
},
{
"name": "lxml",
"specs": [
[
">=",
"4.4.2"
]
]
},
{
"name": "Markdown",
"specs": [
[
">=",
"3.2.1"
]
]
},
{
"name": "msticpy",
"specs": [
[
">=",
"2.3.1"
]
]
},
{
"name": "numpy",
"specs": [
[
">=",
"1.17.3"
]
]
},
{
"name": "packaging",
"specs": [
[
">=",
"19.2"
]
]
},
{
"name": "pandas",
"specs": [
[
">=",
"0.25.3"
]
]
},
{
"name": "python-dateutil",
"specs": [
[
">=",
"2.8.1"
]
]
},
{
"name": "tqdm",
"specs": [
[
">=",
"4.41.1"
]
]
},
{
"name": "python-whois",
"specs": [
[
">=",
"0.7.3"
]
]
},
{
"name": "tldextract",
"specs": [
[
">=",
"4.0.0"
]
]
}
],
"lcname": "msticnb"
}
JSON
