Setup QRadar CE on AWS

 

V0.1a - pfh - 2017/11/26

 

References:

QRadar CE download site: https://developer.ibm.com/qradar/ce/

Jose Bravo's Video tutorials:

Introduction: https://www.youtube.com/watch?v=Ii62Qy3ggnQ

CentOS install: https://www.youtube.com/watch?v=FLMPFH9HgRM

QRadar CE install: https://www.youtube.com/watch?v=i-qA3-b6_ME

QRadar support forum: https://developer.ibm.com/answers/topics/qradarce/

Installing QRadar 7.3.0 on AWS: https://www.ibm.com/support/knowledgecenter/en/SS42VS_7.3.0/com.ibm.qradar.doc/t_Cloud_Install_Qradar_AWS.html

 

Before you start, ensure:

  1. You have an AWS Account - this will not work on Free Tier EC2 - so be prepared to pay Amazon.
  2. You have downloaded the QRadarCE ISO (QRadarCE7_3_0_20171013140512.GA.iso)
  3. You have downloaded the QRadar AWS 'helper' (aws_qradar_prep_1_0.tar.gz) from IBM Fix Central (https://www.ibm.com/support/fixcentral) you will need a (free) IBM ID for this.

 

Step one - Create a suitable Image

 

 

Log into AWS Console and Click on 'Launch Instance'

 

Select the 'AWS Marketplace'

 

In the search bar, type 'centos 7 minimal' and hit Enter

 

Select the 'CentOS 7 (x86_64) - with updates HVM' AMI

 

Review the information and select 'Continue'

 

Select 't2.large' as the Instance Type (this is not free tier eligible)

[I would not recommend anything smaller than a t2.large - but feel free to play]

 

Select 'Next: Configure Instance Details'

 

Choose all default values EXCEPT 'Auto-assign Public IP' which must be changed to 'Disable'

[You may be able to play with these options, but make sure that the Public IP is not Auto Assigned]

 

Select 'Next: Add Storage'

 

Change the 'Root' volume to 80Gb of General Purpose SSD (GP2), check 'Delete on Termination'

 

Select 'Add New Volume'

 

Set the 'EBS' volume /dev/sdb to 100Gb of General Purpose SSD (GP2), check 'Delete on Termination'

[If you want to change these values, don't go any lower.

/dev/sda MUST be at least 78Gb or QRadar will not install.

Using 100Gb for /dev/sdb will leave about 30Gb for the Ariel /store partition on /dev/sdb.

To get more event storage space, it's the /dev/sdb you need to make larger.

You MUST have at least 2 disks]

 

Select 'Next: Add Tags'

 

(You can add tags here if you like - I didn't bother)

 

Select 'Next: Configure Security Group'

 

Set the name to 'SecGrp - qradarCEaws', add a sensible description. Configure the Rules as shown below:

[Again, you may well wish to setup your Security Group differently - this is what I need for my testing, where all my logs are being replayed from my home machine]

 

Select 'Review and Launch'

 

Review the information and select 'Launch'

 

Create a new Key pair and download them

Save the PEM file on your machine. Follow the AWS guidance for conversion and usage. If using PuTTY on Windows, you'll need to convert it with PuTTYgen to a PPK file.

 

Select 'Launch Instances'

 

After this screen spins for a while

 

Scroll to the bottom and select 'View Instances'

 

Observe the instance with no Name and no Public DNS (IPv4)

 

Set a name - e.g. 'qrceaws'

 

Select 'NETWORK & SECURITY' - 'Elastic IPs' From the left of the screen

 

Select 'Allocate new address'

 

Select 'Allocate'

 

Select 'Close'

 

Select the new Elastic IP

 

Select 'Actions' - 'Associate address'

 

Select the Instance and Private IP to use

 

Select 'Associate'

 

Select 'Close'

 

Select 'INSTANCES' - 'Instances' from the left

 

Observe our Instance now has an IP address

 

Select the Instance

 

Copy the Public DNS (IPv4) - bottom right - to the clipboard

 

At this point, you can SSH to the EC2 instance using the keypair saved and the DNS name copied. The userid is 'centos'

 

Step two - Prepare the system

 

I choose to have a small Ubuntu image up on AWS which is storing the ISO and other files, so I don't need to load them up to AWS the whole time. To copy the files across I do the following:

On the centos (QRadarCE) system, find the IP address

$ ifconfig

 

Then update the Security Group protecting the Ubuntu system to allow SSH from the IP address returned

Create an SSH keypair

$ ssh-keygen

Accept all the defaults, then get the contents of .ssh/id_rsa.pub an stick them into .ssh/authorized_keys on the Ubuntu box

 

Use SCP to get the files into /tmp on the centos box with something like:

$ scp ubuntu@172.31.10.180:/home/ubuntu/QRadar/* /tmp/.

 

Check the ISO isn't broken

$ openssl sha -sha256 /tmp/QRadarCE7_3_0_20171013140512.GA.iso

SHA256(/tmp/QRadarCE7_3_0_20171013140512.GA.iso)= d8a2295f71a8733a56fef6a6f0bbd39302d71f12c71487deadb074a7aa93e6b3

 

Extract and run the helper - this will reboot the machine

[centos@ip-172-31-26-109 ~]$ pwd

/home/centos

[centos@ip-172-31-26-109 ~]$ tar zxvf /tmp/aws_qradar_prep_1_0.tar.gz

aws_qradar_prep.sh

[centos@ip-172-31-26-109 ~]$ chmod +x ./aws_qradar_prep.sh

[centos@ip-172-31-26-109 ~]$ sudo ./aws_qradar_prep.sh -install

INFO: Discovering non root devices that can be partitioned...

INFO: Found 1 devices.

INFO: Installing lvm tools...

INFO: Partitioning /dev/xvdb and creating LVM groups.

INFO: Making directories and mounting...

INFO: Disabling SELINUX and external repos...

INFO: Enabling root login and ssh password authentication.

INFO: Rebooting to completely disable SELINUX.

PolicyKit daemon disconnected from the bus.

We are no longer a registered authentication agent.

 

 

When the system has rebooted, connect to it again and continue

We need to put the CentOS repos back into yum, otherwise QRadar installation will fail

$ sudo cp -r /etc/yum.repos.d.old/* /etc/yum.repos.d/.

 

Check all is OK by installing 'screen'

$ sudo yum -y install screen

(if you forget this, then the 'setup' script will do it anyway - but it's a good test to check the repos are there and working)

 

Mount the ISO

[centos@ip-172-31-26-109 ~]$ sudo mount -o loop /tmp/QRadarCE7_3_0_20171013140512.GA.iso /media/cdrom

mount: /dev/loop0 is write-protected, mounting read-only

 

Step three - Install QRadar CE

 

Run the setup command

$ sudo /media/cdrom/setup

 

Read and accept the License Agreement

About to install QRadar Community Edition version 7.3.0.20171013140512

Do you wish to continue (Y/[N])?

'Y' and Enter

Installation will scroll past (with some errors visible) for approx. 15 minutes

 

The next few screens have a 'quirky' user interface. The TAB key is the one to use if in doubt.

Next

Next

Choose 'normal' and Next

Enter an NTP server (strongly recommended) and Next

IMPORTANT ... DO NOT LEAVE THIS ON 'UTC' - it will fail later.

Choose 'GMT' and Next. You _may_ select a real timezone, but that might fail too. There is a post about this on the developerworks forum.

Next

Next

Keep pressing TAB until you get to Next

You do not _need_ to do this here, but (a) it's good practice and (b) if you don't, you'll be forced to change your password on first logon to the GUI.

Enter the password twice and Next

It looks like the same screen is presented again - it isn't. The first one (Admin) is the 'admin' user for logging into the GUI. The second (Root) one is for the system's 'root' user. The Root one defaults to being the same as the admin one, but you can make it different if you like.

Finish

 

This will start installing more stuff - for about 30 minutes, then the system will reboot itself:

 

To log into the GUI, you'll build the URL like:

https://ec2-35-177-57-157.eu-west-2.compute.amazonaws.com/console, where the bit in the middle is specific to your Elastic IP. You'll need to accept the self-signed certificate.

 

If you see this:

Then go grab a coffee until it looks like:

 

Log in with username 'admin' and the password setup above.

 

Step four - Basic Post-Install tasks

As we are running QRadar in an environment it isn't really designed for (I'm assuming you are not going to leave this running 24 x 7 x 365), we can help it out by configuring a few things

Also, some of the defaults are, IMHO, sub-optimal.

 

Update the default retention bucket so we don't kill ourselves with our tiny disk

'Admin' tab -> 'Data Sources' -> 'Events' -> 'Event Retention'

Select the 'DEFAULT' line and 'Edit'

Update as below and Save

And 'Save' again - then 'Close'

(If you like, do the same for Flows - but we aren't actually setting up flows - so that won't matter)

 

Disable auto-updates

'Admin' tab -> 'System Configuration' -> 'Auto Update'

 

Change Settings

 

Set 'Frequency' to 'Disabled'

Save and close the window

 

Disable Coalescing

'Admin' tab -> 'System Configuration' -> 'System Settings'

Change to 'Advanced' view

 

Under 'System Settings' Change 'Coalescing Events' to No

 

Disable GUI timeouts

Also in System Settings, under 'Authentication Settings' - set the 'Inactivity Timeout' to 0

 

While we are in 'System Settings' Consider

·        Setting up SNMP

·        Including headers in CSV exports

·        Enabling X-Force Threat Intelligence Feed

 

Save and close the window

 

When you see the yellow flash - Deploy Changes

 

 

Disable scheduled reports

 

Select the 'Reports' TAB

Select all reports which are not 'Manual'

Actions -> Toggle Scheduling

 

Add a test 'linux' log source and test replayed events to it

'Admin' tab -> 'Data Sources' -> 'Events' -> 'Log Sources'

 

Add

 

Save, Close Window and Deploy Changes

 

Now, go to the 'Log Activity' tab:

Check you can see events going past (i.e. things are working)

Then 'Add Filter'

 

Set as below:

 

Click 'Add Filter'

The events should stop.

 

Reselect 'Real Time Streaming' to clear the view

 

 

Now, log into the QRadar console via SSH

 

[centos@ip-172-31-29-93 ~]$ export user=$RANDOM && echo "<86>$(date +"%b %e %H:%M:%S") testux sshd[$$]: Accepted publickey for U${user} from 192.168.99.99 port 1234 ssh2" >/dev/udp/127.0.0.1/514 && echo U${user}

U29989

[centos@ip-172-31-29-93 ~]$

 

Observe the event on the Log Activity

Or do the same using CygWin on the Windows machine:

$ export user=$RANDOM && echo "<86>$(date +"%b %e %H:%M:%S") testux sshd[$$]: Accepted publickey for U${user} from 192.168.99.99 port 1234 ssh2" >/dev/udp/35.177.57.157/514 && echo U${user}

U32673