Has this ever happened to you?
You get a call from someone within the company asking “Are we monitoring those new systems that got deployed recently?”.
You provide administrators with a Patrol Agent package to install with every new server that is built, and your are suppose to be notified when systems are ready to be configured. The change process involving new servers is clear, and yet you have now clear answer to the question. You fight asking the obvious question “What new servers?”. You quickly rifle through all your emails and change tickets looking for the build notification, and you can’t find any. Your response, “I’m not sure, let me look into this.”. This initial call is immediately followed by multiple managers and directors all asking you, your team and your manager “Why are we not monitoring these new systems?!”.
If you or your team is in charge of configuring your Patrol Agent infrastructure the chances are almost guaranteed that you’ve had this happen before. Communication breakdowns happen all the time for a multitude of legitimate reasons. Luckily there is a way to prevent this from happening, and I’m about to show you the exact steps you can take, to solve this problem once and for all.
As we discussed in our “Introduction to BPPM Central Monitoring Administration (CMA) Policy Automation” there is a BPPM tool available that can make these configuration deployment problems a thing of the past. With this tool you have an amazing array of capabilities that will allow you to fully automate tasks that are performed manually today. This post is the fourth such write-up covering the abilities of the BPPM Central Monitoring Administration (CMA) Console and the use of BPPM Monitoring Policies. In this write-up we are going to show how you can automate the deployment of Patrol Agent configurations to newly built servers in a fully automated manner using your existing PCM configuration files.
First, lets talk about how most BMC shops configure Patrol Agents on new systems today.
Involving The Patrol Agent Silent Installation Package (SIP)
What is the common practice for deploying Patrol Agents today, and how do the agents get properly configured? At the heart of any Patrol Agent deployment process there is a “Silent Installation Package” (SIP), which consists of a bundled Patrol Agent package and probably some associated Knowledge Modules (KMs). This SIP is placed onto a server and when run, will install the agent and/or KM components that are within it.
These packages are most certainly created by the monitoring administration team who is familiar with the Patrol tool, it’s versions and what KMs need to be incorporated into each package. There are silent install packages for agents only, OS KMs, database KMs, patches, etc.. The person building the packages must be knowledgeable of the BMC products in order to ensure it meets the monitoring needs of the business once installed. This is why it best to have a BMC subject matter expert construct these packages and prevent reinstalls later down the road. This is a manual process at the moment, but that will change very soon, and I’ll go into it in a future blog write-up. So stay tuned for that down the road!
Once the packages are created they are usually handed off to system administration teams to be used as part of their server build process at the time of server construction or imaging or deployed using a software distribution tool once the system(s) are on the network. One way or another the silent installation package will be placed onto or within a new server and installed.
The process of installing the Patrol Agent up to this point is normally functional. Getting the agent onto a system can be done at any point before the system is up on the network and considered live. It is at this point however when the first “GOTCHA” can come into play, and that is in the initial configuration of the Patrol Agent.
Manually Configuring a Patrol Agent the Old Way
In order for a Patrol Agent to monitor anything it must be given instructions in the form of “Configuration Rules”. Without them an agent can run as a service on a system forever, just sitting there performing no action at all. It serves no monitoring function at all until you tell it what you want it to monitor, how you want it to alert and where you want it to send events and possibly data up stream into BPPM. Without these instructions, there is no monitoring.
This is normally where the phone and email tag process begins, and the fist point of failure can occur. With so much time on everyone’s hands these days, sitting idle waiting for email notifications or calls from other teams, it’s hard to imagine how this process could fail. (HEAVY SARCASIM HERE) Sadly without a form of configuration automation, the process to properly configure a newly installed agent is fully manual.
Assuming you get the call or email as needed, the BMC SME must go into Patrol Configuration Manager (PCM) and manually add the newly built server and agent to the Patrol Configuration Manager (PCM) application. This is where the deployment of specific Patrol configuration rules get deployed. If the server is Windows 2008 running MSSQL in Development it gets one set of configurations. If it’s Red Hat 5.x running Oracle in Production it gets a certain set. On and on it goes. Within PCM you add the agent to every group associated with the system and the patrol configurations get deployed one by one. All manual!
If for some reason you fail to get the email or phone call, then guess what happens. The system gets put onto the network and the Patrol Agent sits there doing nothing. No monitoring of any kind taking place or generation of any alerts/events at all. The hot seat warms up and everyone takes a turn explaining to the business what went wrong. NOT FUN AT ALL.
Understanding and Using BPPM Policy Selectors to Automate Patrol Agent Configuration Deployments
If that story above strikes a little too close to home, then you are going to LOVE what I’m about to show you. That scenario above can now become a problem of the past!
Imagine setting up a process that would see exactly when a Patrol Agent comes online and immediately deploys the appropriate agent configurations to it. No matter what time of day, with no need for any manual involvement? No phone calls or emails needed to start or delay the process. No more worries about sending the wrong configurations down to a system for any reason. And best of all, a few less hot seats and spot lights to endure in your weekly workload.
That is exactly what the BPPM Central Monitoring Administration (CMA) Policies are now able to perform, automatically.
Using a BPPM CMA Policy you can structure automated configuration deployments out to Patrol Agent that are specific to any of the following selector items. A selector is the thing that the BPPM policy uses to deploy a Patrol Agent configuration. The “Selector” is the item you can use to say when a Policy configuration should be deployed.
These are the BPPM 9.5 Policy Selectors you can use in the BPPM 9.5 CMA Policy Console, and how you could use them.
- Hostname – useful if you have any sort of naming convention tied to your system naming conventions, such as prodxxxx, or windevxxx, which need certain Patrol configurations.
- IP address – useful if your IP address spaces are specific to environment, purpose or OS, which need specific Patrol configuration
- Patrol Agent Port – useful if you have HA structures in your environment using floating Patrol Agents running on unique ports
- Patrol Agent Version – helping you push out specific configurations to new or old agents as needed without mixing the two sets up
- OS version – Still running old legacy systems that you have to treat very specially, this can take care of that, as well as the newest systems.
- BPPM Tags – What I call the Master Selector. This is the item I will cover below. The BPPM tag is a simple agent configuration variable that has amazing capabilities which, I’ll use in this post.
- BPPM Integration Service – setup policies that are specific to certain BPPM integration servers and treats all associated Patrol Agents in a specific way, such as staging or pre-production.
- BPPM Server – lastly the ability to specify Patrol Agent configuration variables that might be BPPM server specific. Very useful in a multi-server environment!
You are now able to construct a policy using any of those items to engage a Patrol Agent configuration deployment (BPPM Policy). If you have a system that is Windows 2008 which has a development IP address, and monitoring a Windows Cluster, you can create 3 different automated BPPM Policies based on the selectors above to keep that system configured a specific way. Better yet, you can use the configurations you already have within PCM to get this done.
Using a BPPM Tag with my BPPM Policy for New Patrol Agent Systems
Now I’m going to show you an example of exactly how to do this using a Patrol Agent SIP. I’m going to show you how to construct a policy that looks for a new “Windows 2008 server” that has just been built, had a Patrol Agent SIP installed on it, and plugged into the network. When it comes onto the network it will get a specific Patrol Agent configuration immediately. And to make it even simpler, I will use an existing PCM rule set that I already use for setting up new Patrol Agents. I will then verify the agent is properly configured AUTOMATICALLY using a BPPM policy.
Here is what I’ll use.
- An unmonitored Windows 2008 server that is freshly imaged with NO Patrol Agent installed.
- A Patrol Agent SIP which consists of the Patrol Agent and Windows OS KMs
- An existing PCM ruleset that would normally be deployed manually to the agent when it is brought online
- A BPPM Tag as my BPPM Policy selector component
The first few items are no different from what you already use in the process of Patrol Agent deployments today. A Patrol SIP deployed to a system to be monitored and a PCM ruleset to be manually pushed out. The little item that changes everything is the little understood BPPM Tag.
You’ve probably seen this screen when creating your Patrol SIPs but never used it.
That little box is now the key to automating your entire Patrol Agent configuration deployment structure. It has an amazing power and it’s been sitting there this entire time.
You can also put one or more entries into that box at the moment you construct the SIP, in this format.
BPPM Tag Examples:
Win2012:”Windows 2012 System”
MSSQL:”Windows MSSQL Server”
RH63:”Redhat 6.3 Server”
OracleProd:”Production Oracle Server”
To put more than one into the SIP you would simple separate them with a comma like this:
Win2012:”Windows 2012 System”, MSSQL:”Windows MSSQL Server”
Here is an example of an actual BPPM Tag from a Patrol Agent configuration for reference, to show you exactly how one looks like to a Patrol Agent and to PCM. There is nothing special about them, AND they can be manually created and deployed using PCM for all existing systems. Think about that after you finish reading this post.
When you embed a BPPM Tag into a Patrol Agent SIP, any BPPM CMA Policy can use it to engage a configuration deployment, which is what we will show.
Note: The only requirement to this is that the Patrol Agent(s) you intend to work with need to be bound to a BPPM Integration Server first, and part of a BPPM infrastructure. The Patrol Agent must check into BPPM first to receive any BPPM policy. If the Patrol Agent isn’t able to reach BPPM, then it will not be able to receive any policy push out from BPPM. Policies are pushed out from the BPPM server, through the Integration Service. Therefor a network connection to a BPPM integration Server is required.
Once you place a tag into a Patrol Agent SIP, and that agent comes online and connects to BPPM, it will immediately receive any CMA policy you have constructed with the BPPM Tag as a selector.
If you’re thinking “Well that would have been great to know before we created all those Patrol Agent and KM SIPs!”, don’t worry. If you want to know how to update all of your packages, be sure to check out the “Bonus” section at the bottom of this post.
Building the BPPM CMA Policy for Patrol Agent Configuration Automation
At this point you have a Patrol SIP with your agent, KMs and a BPPM Tag embedded together. Now it’s time to take the PCM rules that you want to engage with these specific BPPM tags, and create a policy using them.
The BPPM CMA Policy Console interface is located in the web based BPPM web console.
This is the interface that you use to look at the administrative functions of BPPM. Once you login to the admin interface with administrative rights, you should see a “Policies” drawer on the left-hand side. Click on it, and once expanded it should look like this.
Once you expand the Policies drawer, you will see three subfolders exactly like the ones shown above.
- Monitoring – These will hold the seven main CMA Policies.
Note: We introduced these to you in our “Introduction to the CMA Policies” blog post and video presentation. Be sure to check that out after reading this post.
- Blackout – This policy folder holds all of your Patrol Agent blackout policies. We also demonstrated that wonderful ability here, “Putting the Maintenance Blackout Policy to Work”, which is very much worth your while if you find yourself struggling with Patrol Agent maintenance windows and the need to Blackout your Data, Event and/or Recovery Action settings. VERY VERY useful and A MUST See video.
- Staging – this Policy folder is used for pointing your Patrol Agents to specific Integration Server for it’s initial “phone home” at startup. We’ll cover this in a future blog post to fully detail how this could be used.
To setup a New Server Build Policy, we will go into the Monitoring Policy folder.
To construct a new policy simple click on the green plus sign in the upper right hand corner, and then follow along with my demonstration.
After you click on the button, the first selection you are presented with is which type of new CMA Policy you want to construct, which looks like this.
For this demonstration I am going to use the “Configuration Variables” policy. This policy type is very important in order to perform the function I’m going to describe. To read some examples of what each of these seven BPPM Policy Types can do, go here.
Once you are into a new policy you are presented with this popup, to allow you to name and describe your policy. You provide a name you will remember to be associated with the function of the policy and then a description.
Note: This is also where you can “Enable” or “Disable” a policy. I recommend you NOT ENABLE any new policy until you are certain you want it engaged and have tested it out!!
I have provided the name “Win2k8_NewBuild_AutoBaseline” so that I can tell at a glance what this policy is meant for, from the main BPPM Policy page. You should provide a good description that will help you and your team down the road, determine what the policy does without going into it and looking at all the details. So be wordy here! I provided this description to mine.
“This policy is for all servers with a “Win2k8_Build” BPPM Tag. Any server checking into BPPM with that tag will be baselined automatically”
This CMA policy is meant for brand new Windows 2008 servers to configure them with an initial patrol agent “baseline” set of configuration values. DO NOT confuse the term “baseline” here for the BPPM Analytics Baseline. The two are NOT the same.
The next screen is where you construct what is known as the “Policy Selector Criteria”. The selector criteria are the condition(s) that must be met before the policy is engaged, and Patrol Agent configuration data is sent out. For this demonstration I will supply only 1 simple, powerful condition, a BPPM Tag selector.
What this means is this Policy selector shown above will “engage” or become active, against any Patrol Agent that checks into BPPM with the tag “Win2k8_Build”. This would be the tag you placed into the SIP image when you constructed it. Like this example from the SIP I created for this demonstration shown here for reference.
Click on the “Next” button displayed in the lower right section of the policy window to move into the next section of the policy construction. This is where we will define what Patrol Agent configuration variables to use with this policy.
Because I selected the “Configuration Variables” type of CMA Policy, I have a very special ability not found among the other seven different types BPPM Policies. You can pull in/import, your existing PCM configuration files!
When you click on this “import” button you are allowed to pull in any of your existing PCM cfg files. This is KEY to this policy. You can import all of your existing PCM rules that you want the associated with automated BPPM policy distributions. You may or may not understand just how powerful that is at this moment, but you will when you start importing massive amounts of config files later. This import feature will save you a tremendous amount of time when you start to automate these BPPM Policy configurations. This allows you to use everything you’ve already setup in PCM. The alternative is to manually enter them all into the BPPM Policy by hand. NO THANK YOU.
For this demonstration I’ve imported an simple Patrol Agent rule set to provide a basic set of configurations for a new Patrol Agent.
Note: The configuration rule set you import must be local to the system you are using to connect to the BPPM admin console. The best approach to this would be to use the same system you have your production PCM rulesets stored on. When you click “import” you can navigate to the local cfg file on your system and then import them.
Here is a view that shows my import navigation down to the specific “Windows_NewBuild_Baseline.cfg” that I pulled out of my PCM.
Understand, if the cfg file you are using is not in a proper cfg format, you will not allowed to select it and it will be greyed out. All of your configuration sets in PCM should be properly formatted to allow for all of them to be imported.
Once the configuration file is imported you will see all of the Patrol Agent variables that are a part of the PCM config, displayed. Like this.
For the sake of keeping this as simple as possible, I have used a very basic PCM ruleset. In it are the following Patrol Agent variables.
- A pointer for any new Patrol Agent to get to the Patrol Central RTServer
- A list of Preloaded KMs that any new Window 2008 Server needs to have loaded for monitoring
- An integration Server (this would be to point any system to a specific BPPM IS system)
- A list of disabled KMs that I want to block from loading
- A NEW BPPM Tag called “Windows_Baselined”
I could also have imported THRESHOLD or POLLING INTERVAL settings, or notification PCM rulesets as well, which is what makes this automation ability so powerful. You don’t have to recreate all of your PCM rules manually. You would simply go into PCM and find all of your linked “Apply on New” rules, and import them into BPPM CMA Policies. A HUGE time saver!!
Click “Finish” and you’re done with the creation of the BPPM CMA Configuration Variables Policy. Once this policy is enabled, it will immediately apply to every Patrol Agent that has checked into BPPM, and any future Patrol Agent that checks in. Remember, the “Enable Policy” is located on the first page of the Policy creation pages, shown here.
How to Verify the Patrol Agent CMA Policy Automation Is Working
Once the Policy is enabled, it takes effect immediately looking at all your Patrol Agents to see if they meet the “Selection Criteria”. The key word there is “Immediately”. This is main reason you have to be careful with BPPM Policies. To be clear, this is the new “gotcha” point, so be aware of it.
To test my policy I deployed a Patrol Agent onto a virtual system with the embedded BPPM Tag “Win2k8_Build”. I took no action other than to install the SIP onto a clean Windows server called “PATROLSYSMAC1”. My expectation is that once the agent checks into BPPM it should be given all of the configuration variables I have associated with the BPPM CMA Policy I just created. It should preload and disable the appropriate KMs, point to an RTServer etc. All automatically.
To verify which systems a Policy is associate with BMC has provided a nice feature within BPPM. Item 2 highlighted below shows a link to my specific BPPM Policy Console that allows me to see which Patrol Agents it is applied to. In the view of “All Monitoring Policies” below, it shows there were no Patrol Agents associated with it before I enabled the Policy and deployed my Patrol SIP onto my test system.
Once I enabled the BPPM CMA Configuration Variables Policy, the only action I took was to install the Patrol Agent SIP onto the system and did nothing else. After a few short minutes the server checked and the policy was engaged, and the configuration variables pushed down to the system. To verify the policy was deployed I rechecked “Applicable Agents” link again, and it showed my test system “PATROLSYSMAC1″ was now associated with the BPPM CMA Policy as expected.
From this point forward, every time a new Patrol Agent is installed anywhere in my environment, and it has a BPPM Tag “Win2k8_Build“, it will receive this baseline configuration immediately. Patrol Agent Configuration Baseline Automation!!
The only thing left to do now is go through your PCM and start creating BPPM Policies. The first place to start is by looking at all of your “Apply on New” PCM rule sets. Ask yourself what selector criteria you could use to automatically deploy this “Apply on New” rule sets, and then start incorporating BPPM Tags into your SIP images, or deploy out BPPM Tags. As promised, I told you I would cover how to adjust all of your existing Patrol Agent SIPs. That’s next.
BONUS Section: How To Update and Use Your Existing Patrol Agent SIPs
How can you use this information to engage the BPPM CMA Policy features with all of your existing Patrol Agent SIP images? DO NOT recreate all of those packages!! Let me repeat that.
DO NOT recreate your Patrol Silent Installation Packages!!!
Within every SIP, is a file called “install.ctl”. If you open that file, you will find an entry which will allow you to update the SIP by placing a single line into it. Here is an example from the actual SIP I created for this demonstration.
Note: This is meant for all of your 9.0 or greater Patrol Agents, which have the ability to use the BPPM CMA Console. It is best to use this with 9.5 Patrol Agents as well due to changes associated with the port structures and many enhancements. This is something Advantis Management Solutions is very capable of helping you out with, should you need assistance.
All you need to do in order to start using this new BPPM 9.5 automation capability is to make sure you place the appropriate BPPM Tag value within the SIPs that your system administrators are using. Here is an example of the format for reference.
[SETP] /BMC/PATROL/CENTRALADMIN/TAG=TAG_NAME:”Description of Tag”
I hope you found this post informative and useful! Now get out there and start automating those Patrol configurations deployments using the BPPM CMA Policy Automation!!!Merznatalia / Depositphotos.com