From Server Architecture to analyze the impact of Slow-Rate DDoS

On my recent article, I’ve shown you some of the tools being used by Hactivist group to attack Hong Kong sites. I’m sure most of you was surprised how effortless it is to run and generate DDoS attack.

Today, I will show you one more main weapon that has been introduced by the attacker called “Torshammer”.

Torshammer is one of the slow-rate DDoS attack tools. As claimed by Hacktivist group, this tool usually efficient and has great impact on most Apache server of any kind. Why apache?

Let’s start to analyze the impact of slow-rate DDoS attack on Apache Server by reviewing its server architecture.

By default, Apache is using thread-based server architectures that handle each incoming connection with a separate thread and dedicated activity.

Two common thread-based server architectures used by Apache Web server are as follows:

• Multi-process architecture with Multi Processing Module (MPM) Prefork
• Multi-threaded architecture with Multi Processing Module (MPM) Worker

1) MPM Prefork (Multi-Process Architecture):

This is a process-per-connection model. It dedicates a process for handing each connection. When using MPM prefork, the main server process forks several handler (child process) processes on start-up. Those handlers will listen for connection and every connection isolated from each other do not share memory.

The large memory footprint as a result of the connection-process mapping that leads to a concurrency/memory trade-off. Thus, it makes it harder to scale the maximum simultaneous connection.

2) MPM Worker (Multi-Threaded Architecture):

This model of apache can serve a large number of requests with less system resources than the prefork model because on this working model a limited number of processes will serve many numbers of requests. In other word, it is thread-per-connection model.

In this model, multiple threads share the same address space. E.g. shared cache for cacheable responses. Compared to multi-process architectures, a thread only consumes limited memory to handle a connection.

I/O Access inside multi-thread server architecture:

Multi-thread server architecture uses a simple I/O access - A synchronous and blocking I/O operation. Operation system overlaps multiple threads to handle concurrent connections where a reasonable amount of CPU-bound operations must be executed.

In most cases, the blocking I/O operation will trigger scheduling and causes the context switch to allow the next thread to continue.However, during heavy loading, every single thread stack for each connection consumes large amounts of memory and constant context switching will cause huge losses of CPU time. As a result, it will increase chance of CPU cache misses.

In most cases, people will reduce the number of threads to improve the performance of every thread that makes it harder to scale the maximum simultaneous connection.

So, what is slow-rate DDoS?


Slow-Rate DDoS Attack:

There are many different type of slow-rate DDoS attack but their main purpose is similar. It causes the web server application threads to await the end of boundless posts in order to process them. This causes the exhaustion of the web server resources and causes it to enter a denial-of-service state for any legitimate traffic.

The key of slow-rate DDoS is it generates large number of current connections and maintains it as long as possible.

Let us use Torshammer as example:

Torshammer executes a DoS attack by using a classic slow POST attack, where POST content is transmitted in slow rates under the same session (actual rates are randomly chosen within the limit of 0.1-3 seconds).

It will generate number of HTTP POST request and establish the connection to the server (source from Tor network or attacker’s computer). Every connection will hold around 1000 to 30000 second.

If apache can’t close the connection correctly, it will get a lot of current connection at same time and lot of child processes/thread will be spawned.

Inside of each of connection, it is just only sending some randomize characters to maintain the established connection. As the limit of thread-base server architectures, it can’t maintain too many current connections. When the attack hit the maximum of child process/thread that apache can open, it will no longer serve the legitimate traffic.

This tool does not need to generate huge attack bandwidth and high HTTP request per second. Just need to hold the maxim current connection that apache can handle between 1000 to 30000 second. It uses limited resources to terminate the apache server.

Attacker just need to use a Tor network from single computer and can automatically generate an effective DDoS attack.


Conclusion:

In conclusion, not only apache, but also most of thread-base server architectures are suffering from slow-rate DDoS attack. Today, DoS attack is against to the vulnerabilities that related to design of network infrastructure, devices, operation system, platform, services, protocol, etc.

Attacker will study your DDoS related vulnerabilities and select the effective one before attack.

E.g. Hactivist group classified targets by web servers on 14 Oct and attack on 18 Oct 2pm:

So, how to harden our system? Before we discuss the protection, we need to understand the nature of each DDoS attack. Let us go through one by one.

For more reading on server architectures: http://berb.github.io/diploma-thesis/original/042_serverarch.html

To be continuous…

Analysis of Cyber Warfare weapons on DDoS - Hong Kong

Introduction:

As you may aware, Hong Kong is currently under severe multiple DDoS attacks.  The attacks can generally be classified as Volumetric and Application attacks. The Volumetric attacks can be TCP, UDP, ICMP or any kind of protocol attacks that generates huge volume of traffic as efficient as possible. 

Over the years, security devices have been made a lot more robust improvements and now usually can cope with these typical volumetric attacks. However, why most of the Hong Kong sites still got impacted by DDoS attacks in spite of the security measures in placed? My collected statistics indicate that the threat is actually an Application attack that looks like legitimate and may bypass mitigation devices.

In this Blogs, we are going to test 2 weapons(WebHIVE and XMLRPC) and analyze the attack patterns, attack logic, impact, risk and solution.

Let’s start to analyze the tools that being used for attacking Hong Kong websites. 

Attack Tool 1: WebHIVE 

WebHIVE is a web-base DDoS Attack tool that contains JavaScript embedded inside a web-link. The benefit of this tool is its ease of implementation. The Founding Attacker can use any free HTML hosting service to hold the attack link. When the site visitors open the link/html file via their computers/smart phones, they can directly generate the DDoS attack via the browser. There is no need to install any plug-in, library, etc. If your browser supports the JavaScript, then anyone can generate the Application attack from his or her computer/smart phone.

The tool released by a Hacktivist group during the attack period is a simple version.  Compare to the original WebHIVE, it is effortless – more simple and efficient. 

Attacker tries to modify an auto-run version.  They can hard code victim’s IP address, utilize URL weaknesses, automatically modifies attack rate and boosts its attack method.

Some attackers created an auto-run links. If people click the link, his browser automatically executes the JavaScript and generates the selected DDoS request with high rate to targeted victim.

After that, they share to anyone who wants to launch the attacks via online social networking websites and instant-messaging service. Finally, a single target recorded over billion of HTTP request per second.

1) Attack Testing and Testing Result:

We downloaded the source code from a social networking website as following:

When we open the html file or link, it will generate DDoS attack to selected target.

When we open the source code, we find a read me as shown below. This is a sample readme files found inside the HTML:

We have tried to implement the source code from Hacktivist group.

How to implement it?  Let us implement this attack together with several steps and discuss some of our interesting findings.

Based on the on Chrome development tool running result, we’ve changed the source code as follows: 

   a) Set the Target:

       we can set the victim URL in source code. For experience DDoS attacker, they can directly hardcode the vulnerability URL inside source code.  For our test case, we’ve changed to our AWS web server.


    b) Set HTTP request pattern: 

        We can set the AJAX here for targeted web application.  For some advanced attack, they can rewrite as follows:

• Define the query string parameter and add more query string parameter against to victim.
• Rewrite the randomise methodology against to victim.
• Add more query string parameter value to empower the attack damage.

As a result, they can empower the attack and exploit the weaknesses of a website.

   c) Change the HTTP request per sec

       By default, it is unlimited.  We have also set to unlimited for this testing.

Let’s see the attack power from each attack host.

Generate the attack by Attack host with Chrome:

We just open the html via the browser. This is sample traffic from attack host 1 with first 5 minutes.

Attack host 1 traffic:

The first attack host generated around 16.9x8kbps.(1 byte = 8 bits) The website is still fine and accessible.

Then we started the attack from attack host 2 and traffic generated around 420x8bps traffic in peak. At this point, we have observed that server could not handle the traffic anymore and suddenly stopped.

When we tried to check on the AWS web server again, the website was not available and could not access anymore.

Attack host 2 traffic:

Attack traffic summary:

Peak Attack traffic: 139 kbps

HTTP request per sec: 116

Result: Webpage became inaccessible after host 2 started the attack.

Ok, let us go through what’s going on from server side.

During the attack period, there were 21 running instance of apache2 service handler caused by the attack and used 94% of server CPU.  

Memory Summary:

	Before Attack	During Attack
MemTotal:	1016280 kB	1016280 kB
MemFree:	80756 kB	174 kB
Buffers:	106640 kB	84 kB
Cached:	588732 kB	441 kB

When we checked on the web server, there were only 21 established TCP connections. 

TCP connections status:

Root cause: 

Web Service unavailability is caused by end-point resource exhaustion.  Memory is exhausted and apache handler could no longer handle new request.  The server response became slow until could not access anymore.

2) Attack Analysis:

   a) HTTP GET request pattern

      WebHIVE will send the HTTP GET request in the same connection, Once victim received the GET request, web server will respond with requested contents.  

Sample captured traffic by the same TCP stream:

     For some pages of the website, there is a keep-alive mechanism to maintain an active connection between client and server.  

     As the result, when the attacker attacks those pages, the connection set to keep-alive, waiting for the response of the client. 

    In certain point, this is also a caused by poor hardening and code review from web server side.

Detail HTTP Header for every HTTP request:

    


   b) Query String in URL 

       As per analysis above, each request has a random query string value and constant message. Randomize query string added in URL in order to thwart caching and force the web server to reply each request.

       For each of HTTP request, the pattern includes the following: 
 GET /?id=<random value>&msg=<constant message> HTTP/1.1

   c) Using the browser’s HTTP header

       Most of DDoS mitigation devices will drop the attack traffic by HTTP header. However, this tool will use the original HTTP header from the browser. If we drop the attack traffic, it will also drop the normal traffic. 

      In certain cases, some of the intelligent security devices auto-generates filter that impacts and drops the false positive traffic.

   d)  Bypass the DDoS mitigation.

        Most of DDoS mitigation devices will use source host verification. When the attacker opens the victim website via browser and opens the target link, the mitigation device will authenticate the client and tagged as legitimate user. On the other hand, the browser helps to do the authentication before the attack such as learning all the correct URL path, cookies, verification JavaScript or even manually pass the CAPTCHA. As the result, most of source host verification is useless for mitigate this attack.

        As illustrated below, the mitigation devices put the attacker session/IP in a white list and send the authorized code then store in attacker’s browser. The attacker generates an attack by opening a new browser tab and run the attack tool then authorized cookies will be automatically included on the coming attack.

GET /?id=1412739384587&msg=We%20are%20Hackitvist HTTP/1.1

Host: <Victim domain>

Connection: keep-alive

Accept: image/webp,*/*;q=0.8

User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36

Accept-Encoding: gzip,deflate,sdch

Accept-Language: en-US,en;q=0.8,zh-TW;q=0.6,zh;q=0.4

Cookie: _pk_id.19F236C2-8EE1-AB85-C47B-E2516D1D5AEC.7cc5=33ee64db654ad0cd.1383814595.4.1385705542.1385131390.; Hm_lvt_afc503f51323bcd69e0fb1bed230c988=1408001287; 

The YELLOW highlighted data is a Mac book chrome’s HTTP header while GREEN highlighted data is the authenticated cookie embedded in the chrome. 

   e) Short Conclusion:

       For this attack, it is clearly shows that attack would not generate high connection and bandwidth BUT generates high rate of http request from a single IP address that would cause memory exhaustion to web server.  This may also cause greater impact to the web server if attacker lowers the http request at the same time huge users (legitimate + attack) accessing the website that makes the malicious traffic even harder to detect.

3) Risk and Impact

   a) It can also be used as a trap for other people site visitors, for example, if people open it via their computer/smart phone, unknowingly generates a DDoS attack.

       The problem is that script kiddies also can follow the simple instruction and do it step by step, they can generate the attack link and spread the links via social networking websites and Instant messaging service.


   b) As learned from other web resources, Attackers/Hackers can also use a persistent XSS vulnerability to inject JavaScript payload into the <img> tag and post in forum or hacked websites. When the regular visitor open the website, his browser automatically executed the injected JavaScript and force it to generate the DDoS attack.


So, anyone checked your website, if your website hacked by Hacktivist group before?

4) Others Information on WebHIVE

One of example is found at pastehtml.com:
















WebHIVE Supported attack functions:
   a) Image HTTP Request

• Image Request

   b) AJAX HTTP Request

• AJAX GET Request (No message)
• AJAX POST Request

  c) Iframe HTTP Request

• Iframe Request

Attack Tool 2: XMLRPC DDoS Attack

11 Oct 19:00, - The Hacktivist group prepared another attack tool to attack Hong Kong Government. 

XMLRPC attack is a method of http reflection attack.

NOTE: Use the collection of vulnerabilities shared in WordPress xmlrpc  file to  make an attack to the target website. (WordPress is a popular web software provider that you can use to create website or blog)



The tool is different from WebHIVE in a way that one client computer can amplify a huge Layer7 DDoS Flood attack depending on the lists of vulnerable “wordpress site” you have.

1) Attack Testing and Analysis

As learned from other web resources, “any WordPress site with Pingback enabled(which is on by default) can be used in DDOS attacks against other sites. “

Note that XMLRPC is used for pingbacks, trackbacks, remote access via mobile devices and many other features you’re likely very fond of. But, it can also be misused like what we are seeing.

From some hacker forums, people already provided a list of vulnerable WordPress servers as shown below:

2) Attack Logic and Attack analysis

On our testing, we have attacked our AWS web server with 956 WordPress server.  On these lists, there were 192 wordpress servers still active – can pingback and forward the attack.

Sample command line for XMLRPC.exe






As sample below, we have sent out 10 HTTP requests from each WordPress site to AWS web server.
 





10 HTTP request from Attack host to every WordPress site.
  









The tool sends a pingback request to wordpress server then calls “wp_remote_fopen” to download the referring URL. If the wordpress servers has “allow_url_open” activated, this request will try to download the /whole/URL without any timeout or size limit.

Sample POST request from XMLRPC.exe:
The YELLOW highlighted data is a WordPress "Patsy Proxy" site while GREEN highlighted data is the target/victim website.  

POST /xmlrpc.php HTTP/1.1
User-Agent: Opera/9.70 (Linux i686 ; U; zh-cn) Presto/2.2.0
Content-Type: text/xml
Host: kerchin.com
Content-Length: 256
Expect: 100-continue
Connection: Keep-Alive

<?xmlversion="1.0"?><methodCall><methodName>pingback.ping</methodName><params> <param><value><string>http://<victim URL> </string></value></param><param><value><string>http://kerchin.com/?p=5848</string></value></param></params></methodCall>



Capture from AWS server side:

Wordpress servers forward the HTTP GET request to AWS web server.















It is GET request from WordPress server with a special User-Agent. The WordPress version and WordPress domain will also be shown.

GET / HTTP/1.0
User-Agent: WordPress/3.4.2; http://kerchin.com
Host: <victim domain>
Accept: */*
Accept-Encoding: deflate;q=1.0, compress;q=0.5




Other test method:

Simple Curl with script, also can do it:
$ curl -D -  "www.anywordpresssite.com/xmlrpc.php" -d '<methodCall><methodName>pingback.ping</methodName><params><param><value><string>http://victim.com</string></value></param><param><value><string>www.anywordpresssite.com/postchosen</string></value></param></params></methodCall>'

 

3) How do I know if my “wordpress site” is vulnerable or not?

We can check our wordpress server on the following link:
http://labs.sucuri.net/?is-my-wordpress-ddosing

Appendix:

Testing environment:

We have set-up the testing environment as follows:










Attack host: 
MacBook Pro x2
Processor: 2.5GHz Intel Core i5
Memory: 4GB


Victim Web server:
Standard AWS Web server
Processor: Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
MemTotal: 1016280 kB
MemFree: 80756 kB
Buffers: 106640 kB
Cached: 588732 kB
Server: Apache/2.4.7 (Ubuntu), PHP/5.5.9-1ubuntu 14.4
Application: Wordpress 4.0


Reference:
http://www.incapsula.com/blog/world-largest-site-xss-ddos-zombies.html
http://chr13.com/2014/04/20/using-facebook-notes-to-ddos-any-website/
http://blog.spiderlabs.com/2014/03/wordpress-xml-rpc-pingback-vulnerability-analysis.html

http://blog.sucuri.net/2014/03/more-than-162000-wordpress-sites-used-for-distributed-denial-of-service-attack.html