ServiceNow - Discovery Integration

ServiceNow - Discovery Integration

1. Objective

This Technical Design Document includes details like overview of Service-Now, architecture diagram, Component, scope, Configuration of Auto Discovery and basic standard operating procedure For Service-Now Discovery.

The purpose of this document is to enable the user to effectively use the Service-Now Discovery tool & running Discovery on Servers & Network Devices.

2. Overview

Service Now offers on-demand IT service management. This software as a service (SaaS) platform contains a number of modular applications that can vary by instance and user.

Discovery uses conventional techniques and technology to extract information from computers and other devices. It uses a wide variety of probes (simple commands or queries) to gather information, and matching sensors (small, simple programs, usually in JavaScript that we can modify) to analyse that information and load it into the CMDB. Discovery uses these probes and sensors to explore any given computer or device.

For each type of device, Discovery uses different kinds of probes to extract more information about the computer or device, and the software that's running on it:

ØWindows computers and servers: remote WMI queries, shell commands

ØUNIX and Linux servers: shell command (via SSH)

ØNetwork gear (switches, routers, etc.): SNMP queries

2.1 Logical Architecture Diagram 

2.2 Key Design Drivers 

Following are the key design drivers for Discovery solutions

The information that Discovery gathers about devices can be used to update the Configuration Management Database (CMDB) automatically. Discovery employs Identifiers to search the CMDB for Configuration Items (CI) that match devices discovered in the network.

ØThese Identifiers can be configured to instruct Discovery to take certain actions when device matches are made or not made. There are three possible result states that Discovery recognizes:

1. When a discovered device is found to match an existing CI in the CMDB, then continue Discovery and update the CI

2. When a discovered device is not found to match an existing CI, then continue Discovery and create a new CI in the CMDB

3. Take no action in the CMDB, whether a match is made or not. Discovery stops after the identification process.

2.3 Service Now Components description

The MID Server is deployed in the internal network, so it has access to connect directly to discoverable devices (orange lines). Typical protocols the MID Server uses to communicate with devices are:

ØUDP

ØSNMP (Routers / Switches)

ØTCP

ØSSH (Secure Shell, UNIX)

ØWMI (Windows Management Instrumentation)

ØWindows PowerShell 

2.4 Data Collection and Processing (Probes and Sensors)

At Service Now, refer "Data Collection" as a Probe and "Data Processing" as a Sensor. Hence refer the term "Probes and Sensors" as "Data Collection and Processing". The purple lines represent the MID Server probing for information. The MID Server then passes the results of each probe to the Service Now instance via SOAP/HTTPS (green lines). Sensors (red box) within the Service Now instance process the information gathered and update the CMDB.

2.5 Components

ØMID Servers - The Management, Instrumentation, and Discovery (MID) Server is a Java server that runs as a Windows service or UNIX daemon. The MID Server facilitates communication and movement of data between the Service Now platform and external applications, data sources, and services.

ØProbing the Neighbourhood - The MID Server is out in the network (neighbourhood) using probes (census agents) to gather information. That information is returned to the instance where the sensor processes the data. Sometimes, when the MID Server passes the information to the sensor, the sensor decides that it needs more information and makes the MID Server run more probes.

The following diagram demonstrates the conversation between the MID Server and the sensor:

2.6 Service a/c

A service account is just a user account that is used to run some sort of services. It is common to have service account passwords not expire, as that can be difficult to manage since no user will receive a warning about the password expiring during login. Service accounts are typically not used to log on interactively to a machine, although since they are a normal user account, it is possible.

For SNOW Discovery prospect, Service a/c is used to run remote queries on target devices.  SNOW Discovery requires login authentication on target host to run its queries. SNOW Discovery stores these credentials in an encrypted field on the Credentials table, and once they are entered, they cannot be viewed.

Separate Service a/c are used for Separate devices & environment

ØDomain-wide Windows Service a/c for Domain-wide Windows Servers

ØLocal Windows Service a/c for Non-Domain Windows Servers

ØLocal Service a/c for Unix Servers

ØSNMP(read only) Community String for Network Devices

3. Assumptions, Scope and Risks

3.1 Assumptions

ØAll the prerequisites for Service Now will be fulfilled.

ØAll the required information about existing IT infrastructure for implementing Service-now Discovery will be provided by Capsugel

ØRequire access will be provided for discovery 

3.2 Scope

ØInstallation and configuration of MID Servers

ØConfiguration of Discovery

ØDiscovery on Windows Servers & Network devices.

3.3 Risks

ØDelay in provisioning of information

ØDelay on Data Gathering which can also lead to overall delay in project implementation.

4. Using Service-Now Discovery

This section describes how to get started with Service-Now Discovery, how to Install & configure MID Server.

4.1 Login into SNOW Discovery UI

· To access the UI, enter the following URL into browser:

Example: https://xyz.service-now.com

(The login page is displayed)

· Enter credentials in login page & login

(The main Service-Now Home page is displayed)

· Type Discovery in Type filter text Box (marked with red circle) & it will show Discovery menu

4.2 Logging Out from UI

To ensure system security, always log out when leaving the system. Click the Logout link on the right-hand-side of the primary navigation bar which is displayed on each page:

4.3 Installation & configuration of MID Server

The MID Server must be able to communicate with the machines it is configured to probe. Ensure that the system on which the MID Server is installed has the following network privileges:

· Configure any firewalls between the MID Server and the target devices to allow a connection. If network uses a DMZ, and if network security protocols limit port access from within the network to the DMZ, deploy a MID Server to a machine within the DMZ to probe the devices there

· Configure target devices to allow the MID Server probe to connect

· Install the MID Server with the proper account: either local or domain administrator

· Logon to the host system on which installation of the MID Server requires

· Login into SNOW Discovery UI into browser

· Type MID Server in type filter text box(as shows in pic)

· Click on Downloads & Select and download the MID Server for the operating system targeting in the form that appears

· Create a directory for the MID Server on the top level of the drive with a distinctive name, such as ServiceNow\MID Server1

· Extract the downloaded MID Server archive file into new directory. Right-click the package and select Extract All from the pop-up menu

· Navigate to the \agent directory that was created when the file was extracted.

· Edit the config.xml file using a text editor such as WordPad & make below changes:

ØFind the element <parameter name="url" value= "https://YOUR_INSTANCE.service-now.com" /> and change the value to the URL

ØFind the element  <parameter name="mid.instance.username"  value= "YOUR_INSTANCE_USER_NAME_HERE" /> and change the value

ØFind the element  <parameter name="mid.instance.password" value= "YOUR_INSTANCE_PASSWORD_HERE" encrypt="true"/> and change the value

ØFind the element <parameter name="name" value= "YOUR_MIDSERVER_NAME_GOES_HERE"/> and change the value

ØSave & exit from config.xml file

  

· Install the MID Server as a Windows service

ØClick the Start button

ØIn the search box, type command prompt or cmd.exe

ØIn the result list, right-click Command Prompt or cmd.exe, and then click Run as administrator

ØIn the command prompt, navigate to \agent in the directory  created for the MID Server files

ØRun start.bat

· Edit the MID Server's credentials:

ØOpen the Windows Services console.

ØDouble-click on the Service-Now MID Server service.

ØIn the properties dialog box, select the Log On tab.

ØSet Log on as privileges with Domain User or Local Admin credentials.

ØIn the General tab, set Startup type to Automatic.

ØClick OK.

· Re-start the Service-Now MID Server service and make sure that the MID\agent\logs\agent0.log does not have error messages

· On the instance to which this MID Server is connected, navigate to MID Server > Servers. If Discovery is installed, navigate to Discovery > MID Servers (All MID Servers connected to this instance are listed here)

· Make sure that the Status of the MID Server just installed is UP

4.4 Uninstalling MID Server 

The MID Server runs as a stand-alone service and can be removed easily to accommodate such tasks as redeploying the MID Server to another host machine, or changing the unique name of a MID Server when deploying multiple MID Servers. Below are the steps to uninstall a MID Server:

· Stop the running MID Server service

· Open a command window (Start > Run > cmd)

· Go to the \agent\bin folder in the MID Server installation folder

· Type Uninstall.bat & press Enter

4.5 Setting UP Service-Now Discovery for Discovery 

4.5.1 Setting up Credentials

Setup Credentials in SNOW Discovery UI

· Type Discovery in type filter text window

· Go to Credentials click New

· In name field, enter any Unique and descriptive name for this credential

· In Type field, Select between SSH, SSH Private Key, SNMP, Windows, VMware and MSSQL  credentials

· In User Name field, Enter a user name for selected type of credential

· In Password field, Type the password to use. If we have selected SNMP type credentials, type the community string here

· In SSH passphrase field, Type a secure SSH passphrase. This field is available only for SSH Private Key credentials

· In SSH Private Key field, Enter a secure, private key that can be used instead of a password for SSH logins. This field is available only for SSH Private Key credentials

· In Order field, the order (sequence) in which the platform tries this credential as it attempts to log onto devices. The smaller the number, the higher in the list this credential appears

· In Applies to field, Select whether to apply these credentials to All MID servers in  network, or to one or more Specific MID servers

· In MID Server field, select one or more MID Servers from the list of available MID Servers. The credentials configured in this record are available to the MID Servers in this list. This field is available only when select Specific MID servers from the Applies to field

· In Active field, Enable or disable these credentials for use

4.5.2 Credentials used:

ØFor Domain Windows Servers: Corp\SRVGBL-SNOW

ØFor Network Devices(SNMP String): ***********

Note: Passwords & SNMP Community String are not shared due to security reason

4.5.3 Scheduling Discovery

Scheduling Discovery in SNOW Discovery UP 

· Type Discovery in type filter text window

· Go to Credentials click New

· In Name field, Type a unique, descriptive name for schedule

· In Discover field, Select between Configuration Items & IP Addresses

ØConfiguration item scans use Discovery Identifiers to match devices with CIs in the CMDB and update the CMDB appropriately

ØIP address scans discover all the active IP addresses in the specified range and create device history records, but do not update the CMDB

· In MID Server field, Select the MID Server to use for this schedule

· In Location field, pick a location to assign to the CIs that are discovered by this schedule. If this field is blank, then no location is assigned

· In Max run time, Set a limit to the amount of time Discovery can run on this schedule

· In Active field, select the check box to enable this schedule. If clear the check box, the schedule for this action is disabled

· In Include Alive field, Select this check box to include all devices that have at least one port that responds to the scan, but no open ports

· In Log state change field, Select this check box to create a log entry every time the state changes during a Discovery

· In Run Field, Select whether Discovery runs on a periodic schedule

ØIf select Once, Discovery runs one time after an update to the record

ØIf select a Weekly or Monthly schedule, Select the day of the week or month on which Discovery runs in a weekly or monthly schedule

ØIf select periodically schedule, Repeat Interval selection is available. Select the number of days between scheduled Discovery runs

· In Time field, Select the time of day this schedule should run

· Click Submit to save the schedule

Quick Range

· In Quick range field, Define IP addresses and address ranges to scan by entering IP addresses in multiple formats (network, range, or list) in a single, comma-delimited string

Discovery IP Ranges

· In Discovery IP Ranges field, defines the ranges of IP addresses to scan using this schedule

Excluding IP Addresses

 Specific IP addresses in a range or network might need to be excluded from an established Discovery Schedule. This is necessary if a subnet contains critical-use devices that cannot be discovered due to a local policy that prevents interacting with those devices.

· In the Discovery Schedule, click the link for the Type of IP address range that contains the address to exclude

· Click New in the Discovery Range Item Excludes Related List

· In the Discovery Range Item Exclude form, select a Type for the excluded IPs

· Select IP Address List to exclude a single IP address or multiple IP addresses that are not sequential

· Right-click in the record's header bar and select Save from the pop-up menu

· The Discovery Range Item IPs Related List appears

· Click New in this list

· An entry form for the IP addresses or hostnames to exclude appears

· Enter a single IP address to exclude or multiple addresses separated by commas, and then click Submit

· The excluded IP address appears in the Discovery Range Item IPs Related List at the bottom of the Discovery Range Item Exclude form for that IP address Type

· Click Update to save excluded address and return to the Discovery Schedule

4.6 Discovery Status

The Discovery Status interface (Discovery > Status) displays all the details of a Discovery. Each record in the Status list represents the execution of a Discovery by a schedule and displays such high level information as the date of the Discovery, the mode, the number of probe messages sent to devices, and the number of sensor records that were processed 

4.6.1 Discovery Status Record 

By default, only 30 days of Discovery records are displayed in the status list at a time. The status record provides the following fields:

Field	Input Value
Number	Discovery record ID generated by Service-Now instance.
Description	How this Discovery was run. Typically, the description is Scheduled, but if ran Discovery manually from a Schedule, the record would show Discover Now in the Description field.
Schedule	The name of the Discovery schedule.
State	Indicates the state of this Discovery (Active, Completed, etc.)
Started	The number of probe messages sent to devices from the MID Server.
Completed	The number of sensor records that were processed. This number must match the number of probes launched.

4.6.2 Discovery Log

The Discovery Log displays all the activity for this Discovery, including such things as classification failures, CMDB updates, and authentication failures.

Log Information

· The Discovery Log provides the following information:

Field	Input Value
Created	Timestamp of the Discovery activity. Each timestamp defines the approximate time of the activity. Several Discovery events may occur in random order within a second.
Level	Classifies the activity into one of the following levels for general sorting: Error Information Warning
Message	Informative message detailing the outcome of the activity or the Discovery progress. Look here for the result of a classify probe or for authentication failure.
Source	Names the particular activity, such as the Shazzam probe or a UNIX classify probe.
CMDB CI	Names a device for which a matching CI was found in the CMDB. Click this link to drill down into the CI record for the device.
Sensor	Names the sensor that processed the results of a specific probe that was run
Device	Lists the IP address of the CI discovered. All devices identified by IP address appear in the log, even if they refused all invitations to communicate.

4.6.3 Devices

Select the Devices Related List to view a summary for all the devices scanned. During a Discovery, the list tracks current and completed activity and displays an incremental scan counter. When Discovery is finished for a device, the final disposition is displayed in the Completed activity column. Successful Discoveries that result in updated or created CIs are highlighted in green.

· The following information is displayed in this form:

Field	Input Value
Source	The IP address of the device discovered.
Completed activity	Indicates the outcome of Discovery for this device or the last completed activity for a Discovery in progress, such as Identified CI. Successful outcomes are indicated in green.
Current activity	The current scanning activity for this device for a Discovery in progress, such as Updating CI.
CMDB CI	The name of this device as it appears in the CMDB.
Started	The number of device-specific probes runs. This number does not include the universal probes, Shazzam and Ping, that run initially.
Completed	The number of sensor records created from the device-specific probes that were run.
Scan status	Shows the final scan count of a completed Discovery or an incremental scan counter for a Discovery in progress (e.g. Scan 17 of 19).
Issues	Displays the number of issues encountered during Discovery of this device. Select the Discovery Log Related List to view these issues.

4.6.4 ECC Queue

The External Communication Channel (ECC) Queue provides all probe and sensor records for a particular device for a particular Discovery. Drip down in the ECC Queue from the beginning to the end of the Discovery, or drop into the sequence of probes and sensors at any point.