Benchmark with Cosbench

Documentation
Name:	Benchmark with Cosbench
Description:	How to stress ceph with cosbench tool
Modification date :	15/02/2019
Owner:	dodger
Notify changes to:	dodger
Tags:	ceph, oss
Scalate to:	The_fucking_bofh

Intro

Cosbench is a stress tool for object storage systems:
https://github.com/intel-cloud/cosbench It support the main technologies: S3, Swift.

Installation

I found multiple problems trying to run cosbench.
The higher version I was able to run without problems is 0.4

Pre-requisites

Java Run Time Environment 1.6 or later (JDK)
Curl 7.22.0 or later
NC (netcat) package
Unzip package

Fix script

The startup script will not work in redhat/centos 7, you should need to comment/delete the following line:

TOOL_PARAMS="-q 1"

For example:

sed -i '/TOOL_PARAMS/d' cosbench-start.sh

Setup driver

Edit the file:

conf/controller.conf

and change the line:

url = http://127.0.0.1:18088/driver

With the ip address of the server.

Startup

bash start-all.sh

Sample:

bvmld-osl-001 /home/bofher/cosbench/0.4.0.0 # bash start-all.sh                                                                                                                                                 
Launching osgi framwork ... 
Successfully launched osgi framework!
Booting cosbench driver ... 
Ncat: Connection refused.
.Ncat: Connection refused.
..
Starting    cosbench-log_0.4.0.0    [OK]
..
Starting    cosbench-tomcat_0.4.0.0    [OK]
Starting    cosbench-config_0.4.0.0    [OK]
Starting    cosbench-http_0.4.0.0    [OK]
Starting    cosbench-cdmi-util_0.4.0.0    [OK]
Starting    cosbench-core_0.4.0.0    [OK]
Starting    cosbench-core-web_0.4.0.0    [OK]
Starting    cosbench-api_0.4.0.0    [OK]
Starting    cosbench-mock_0.4.0.0    [OK]
Starting    cosbench-ampli_0.4.0.0    [OK]
Starting    cosbench-swift_0.4.0.0    [OK]
Starting    cosbench-keystone_0.4.0.0    [OK]
Starting    cosbench-httpauth_0.4.0.0    [OK]
Starting    cosbench-s3_0.4.0.0    [OK]
Starting    cosbench-librados_0.4.0.0    [OK]
Starting    cosbench-scality_0.4.0.0    [OK]
Starting    cosbench-cdmi-swift_0.4.0.0    [OK]
Starting    cosbench-cdmi-base_0.4.0.0    [OK]
Starting    cosbench-driver_0.4.0.0    [OK]
Starting    cosbench-driver-web_0.4.0.0    [OK]
Successfully started cosbench driver!
Listening on port 0.0.0.0/0.0.0.0:18089 ... 
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 18088 !!!
----------------------------------------------

======================================================

Launching osgi framwork ... 
Successfully launched osgi framework!
Booting cosbench controller ... 
Ncat: Connection refused.
.Ncat: Connection refused.
...
Starting    cosbench-log_0.4.0.0    [OK]
.
Starting    cosbench-tomcat_0.4.0.0    [OK]
Starting    cosbench-config_0.4.0.0    [OK]
Starting    cosbench-core_0.4.0.0    [OK]
Starting    cosbench-core-web_0.4.0.0    [OK]
Starting    cosbench-controller_0.4.0.0    [OK]
Starting    cosbench-controller-web_0.4.0.0    [OK]
Successfully started cosbench controller!
Listening on port 0.0.0.0/0.0.0.0:19089 ... 
Persistence bundle starting...
Persistence bundle started.
----------------------------------------------
!!! Service will listen on web port: 19088 !!!
----------------------------------------------

Stop

bash stop-all.sh

Cosbench url

http://SERVER_IP_OR_HOSTNAME:19088/controller/index.html

Workload config

This is a sample of a HEAVY stress test run over our development CEPH:

workload-config.xml

<?xml version="1.0" encoding="UTF-8"?>
<workload name="s3-sample" description="sample benchmark for s3">
    <auth type="none"/>
    <storage type="s3" config="accesskey=****;secretkey=****;endpoint=http://bvmld-osgw-101.ciberterminal.net"/>
    <workflow>
        <workstage name="init" closuredelay="0">
            <auth type="none"/>
            <storage type="s3" config="accesskey=****;secretkey=****;endpoint=http://bvmld-osgw-101.ciberterminal.net"/>
            <work name="init" type="init" workers="100" interval="5"
                division="container" runtime="0" rampup="0" rampdown="0"
                afr="0" totalOps="100" totalBytes="0" config="cprefix=s3testqwer;containers=r(1,10)">
                <auth type="none"/>
                <storage type="s3" config="accesskey=****;secretkey=****;endpoint=http://bvmld-osgw-101.ciberterminal.net"/>
                <operation type="init" ratio="100" division="container"
                    config="objects=r(0,0);sizes=c(0)B;cprefix=s3testqwer;containers=r(1,10)" id="op1"/>
            </work>
        </workstage>
        <workstage name="prepare" closuredelay="0">
            <auth type="none"/>
            <storage type="s3" config="accesskey=****;secretkey=****;endpoint=http://bvmld-osgw-101.ciberterminal.net"/>
            <work name="prepare" type="prepare" workers="100"
                interval="5" division="object" runtime="0" rampup="0"
                rampdown="0" afr="0" totalOps="100" totalBytes="0" config="cprefix=s3testqwer;containers=r(1,10);objects=r(1,100);sizes=c(64,1024)KB">
                <auth type="none"/>
                <storage type="s3" config="accesskey=****;secretkey=****;endpoint=http://bvmld-osgw-101.ciberterminal.net"/>
                <operation type="prepare" ratio="100" division="object"
                    config="createContainer=false;cprefix=s3testqwer;containers=r(1,10);objects=r(1,100);sizes=c(64,1024)KB" id="op1"/>
            </work>
        </workstage>
        <workstage name="main" closuredelay="0">
            <auth type="none"/>
            <storage type="s3" config="accesskey=****;secretkey=****;endpoint=http://bvmld-osgw-101.ciberterminal.net"/>
            <work name="main" type="normal" workers="200" interval="5"
                division="none" runtime="30000" rampup="0" rampdown="0"
                afr="200000" totalOps="0" totalBytes="0">
                <auth type="none"/>
                <storage type="s3" config="accesskey=****;secretkey=****;endpoint=http://bvmld-osgw-101.ciberterminal.net"/>
                <operation type="read" ratio="60" division="none"
                    config="cprefix=s3testqwer;containers=u(1,10);objects=u(1,100)" id="op1"/>
                <operation type="write" ratio="40" division="none"
                    config="cprefix=s3testqwer;containers=u(1,10);objects=u(101,200);sizes=c(64,1024)KB" id="op2"/>
            </work>
        </workstage>
        <workstage name="cleanup" closuredelay="0">
            <auth type="none"/>
            <storage type="s3" config="accesskey=****;secretkey=****;endpoint=http://bvmld-osgw-101.ciberterminal.net"/>
            <work name="cleanup" type="cleanup" workers="200"
                interval="5" division="object" runtime="0" rampup="0"
                rampdown="0" afr="0" totalOps="200" totalBytes="0" config="cprefix=s3testqwer;containers=r(1,10);objects=r(1,200)">
                <auth type="none"/>
                <storage type="s3" config="accesskey=****;secretkey=****;endpoint=http://bvmld-osgw-101.ciberterminal.net"/>
                <operation type="cleanup" ratio="100" division="object"
                    config="deleteContainer=false;cprefix=s3testqwer;containers=r(1,10);objects=r(1,200)" id="op1"/>
            </work>
        </workstage>
        <workstage name="dispose" closuredelay="0">
            <auth type="none"/>
            <storage type="s3" config="accesskey=****;secretkey=****;endpoint=http://bvmld-osgw-101.ciberterminal.net"/>
            <work name="dispose" type="dispose" workers="100"
                interval="5" division="container" runtime="0" rampup="0"
                rampdown="0" afr="0" totalOps="100" totalBytes="0" config="cprefix=s3testqwer;containers=r(1,10)">
                <auth type="none"/>
                <storage type="s3" config="accesskey=****;secretkey=****;endpoint=http://bvmld-osgw-101.ciberterminal.net"/>
                <operation type="dispose" ratio="100"
                    division="container"
                    config="objects=r(0,0);sizes=c(0)B;cprefix=s3testqwer;containers=r(1,10)" id="op1"/>
            </work>
        </workstage>
    </workflow>
</workload>

You must change:

endpoint
accesskey
secretkey
runtime (in the workstage name=“main” part), with this sample it will run for 30000 seconds.

Ciberterminal Wiki

Table of Contents