SX.enterprise 5.

700 user Load Test Report

with Citrix

Copyright 2007 Infor Global Technology Solutions GmbH and/or its affiliates and subsidiaries
All rights reserved. The word and design marks set forth herein are trademarks and/or registered trademarks of Infor
Global Solutions Technology GmbH and/or its affiliates and subsidiaries. All rights reserved. All other trademarks
listed herein are the property of their respective owners.

Important Notices
The material contained in this publication (including any supplementary information) constitutes and contains
confidential and proprietary information of Infor Global Solutions.
By gaining access to the attached, you acknowledge and agree that the material (including any modification,
translation or adaptation of the material) and all copyright, trade secrets and all other right, title and interest therein,
are the sole property of Infor Global Solutions and that you shall not gain right, title or interest in the material
(including any modification, translation or adaptation of the material) by virtue of your review thereof other than the
non-exclusive right to use the material solely in connection with and the furtherance of your license and use of
software made available to your company from Infor Global Solutions pursuant to a separate agreement (Purpose).
In addition, by accessing the enclosed material, you acknowledge and agree that you are required to maintain such
material in strict confidence and that your use of such material is limited to the Purpose described above.
Although Infor Global Solutions has taken due care to ensure that the material included in this publication is accurate
and complete, Infor Global Solutions cannot warrant that the information contained in this publication is complete,
does not contain typographical or other errors, or will meet your specific requirements. As such, Infor Global Solutions
does not assume and hereby disclaims all liability, consequential or otherwise, for any loss or damage to any person
or entity which is caused by or relates to errors or omissions in this publication (including any supplementary
information), whether such errors or omissions result from negligence, accident or any other cause.

Trademark Acknowledgements
[INSERT NAME OF REGISTERED TRADEMARK] is a registered trademark and [INSERT NAME OF
TRADEMARK ] is a trademark of [INSERT NAME OF COMPANY] in the United States and/or other countries.
All other company, product, trade or service names referenced may be registered trademarks or trademarks of their
respective owners.

Publication Information
Document code: PD #46661
Release: SX.enterprise 5.1
Publication date: March 14

Table of Contents

About This Report.............................................................................................................................iii

1. Summary

1.1 Test characteristics...................................................................................................................1

1.2 Test results...............................................................................................................................2
1.3 Test conclusion.........................................................................................................................2
2. Introduction......................................................................................................................................1
2.1 Project.......................................................................................................................................1
2.2 Participants...............................................................................................................................1
2.3 Acknowledgements..................................................................................................................1
2.4 Project goals and expectations................................................................................................2
2.5 Load Test description...............................................................................................................2
2.6 Other information......................................................................................................................3
3. Hardware setup................................................................................................................................1
3.1 Hardware setup........................................................................................................................1
3.2 Involved hardware....................................................................................................................2
3.3 I/O configuration nlbaupbc5......................................................................................................2
3.4 Database layout........................................................................................................................3
4. Software setup.................................................................................................................................1
4.1 Software versions + patches.....................................................................................................1
4.2 Tuning.......................................................................................................................................2
4.2.1 Citrix Servers....................................................................................................................2
4.2.2 Aix Server.........................................................................................................................2

ii

Table of Contents

4.2.3 Progress OpenEdge........................................................................................................2

5. Test Results.....................................................................................................................................1
5.1 Response times with low load...................................................................................................1
5.2 Citrix server testing...................................................................................................................3
5.2.1 Observations....................................................................................................................4
5.3 Application and DB testing.......................................................................................................6
5.3.1 300 users on application and database server...............................................................7
5.3.2 700 users on application and database server.............................................................10
5.4 Loadfactor model testing........................................................................................................14
6. Conclusions.....................................................................................................................................1
6.1 Responsiveness of SX.e..........................................................................................................1
6.2 Resource consuption per user.................................................................................................1
6.3 Transactions per hour, day for 700 users................................................................................2
6.4 Load test execution...................................................................................................................2

About This Report

This report presents the high level results of the load test of SX.enterprise 5.1 in two-tier mode.
The SX.e clients on Citrix servers and the Application and database server on a IBM AIX
system.
Chapter 1 Summary gives an high level overview of the results
Chapter 2 Introduction is an introduction to the project goals and load test description.
Chapter 3 Hardware setup gives and detailed overview of the hardware setup.
Chapter 4 Software setup shows the used software, applied patched and tuning.
Chapter 5 Test Results presents the results of the most important test results.
Chapter 6 Conclusions draws some first high level conclusions.

1.Summary

This chapter gives the summary of the test setup and the results of the load test.

1.1 Test characteristics

Hardware vendor

Dell

Model

PowerEdge 1950

CPU type / Clock

64-bit Intel Xeon 5150

dual core 2.66 GHz

Cache
L2 /
Memory

4MB / 8 GB

Operating System

Windows 2003

Version

Enterprise, 32-bit

Infor

SX.enterprise

Version

5.1

Citrix

Presentation Server

Version

4.0

Hardware vendor

IBM

Model

P570

CPU type Clock

3 x Power5 1.5 GHz

Dual Core

Cache
L3 /
Memory

36 MB / 20 GB

Operating system

AIX

Version

5.3

Progress

OpenEdge

Version

10.0B.05.03, 64-bits

Infor

SX.enterprise

Version

5.1

Load Test

Infor SX.e

Version

Rel 1_1

Authors

Jeffrey Brown, Erik Remmelzwaal

Date

Xxxxxxxx

Sizing Guide

Xxxxxxxx

1.2

Test results

These are the measured results in number of actual simulated users.

Server

Users

CPU

Memory

Disk

IBM P570

700

80%

16 GB

6 Raid 0

Dell 1950

240

50%

8 GB

1, 15k rpm

Based on the measurements and the presented configuration one can present the number of
transactions per hour and day (24 hours) for the major transactions. This is inclusive OE Order
picking reporting for all entered orders.

AP Costing
AR Cash Receipt
OE Inquiry
OE Order Entry
OE shipping
PO Order Entry
Logon/Logoff

Transactions Transactions Lines [# per Lines [# per

[# per hour] [# per day]
hour]
day]
2,786
66,874
5,251
126,029
698
16,762
1,397
33,523
12,574
301,766
62,868
1,508,832
6,010
144,230
6,010
144,230
1,438
34,502
10,063
241,517
101
2,419

1.3 Test conclusion

The Infor SX.enterprise load test shows linair scalability up to the measured 700 concurrent
users in response times inclusive one report manager running order picking.
The SX.e clients ran on a citrix server and was memory bound at 260 users with 8 GB. Due to
load runner constrains we were forced to work with 32-bits windows. However it is no problem
to switch to windows 2003 64 bits server.

2.Introduction

2.1 Project
Location

Infor, Performance and Benchmark Center

Address

Baron van Nagellstraat 89

3771 LK Barneveld
Netherlands

2.2

Participants

Name 1

Jeffrey Brown

Company

Infor

Telephone

(719) 264-4752

E-mail
Name 2

Erik Remmelzwaal

Company

Infor

Telephone

+31 342 428923

E-mail

2.3

Acknowledgements

Special thanks to Intel who provided two loaner systems. Also special thanks to Citrix who
donated 700 user license for free to accomplish this benchmark and to Martijn Bosschaart from

Citrix who joined us one day to check and tune the Citrix Pressentation farm setup. Special
thanks to various Infor collegues that assisted with advice, scripting and even temporarily giving
up there project to help to fulfull the hardware needs for this project.

2.4

Project goals and expectations

The goal of this project is to deliver sizing information for Infor SX.enterprise based on the
results of controlled load testing. This includes sizing for the OpenEdge database server with
SX.e Application server and Citrix server hosting the SX.e clients.
This information is presented on Infors customer event Inforum in September 2007.
The project consists of running up to 700 users during a load test. This load test is the first one
executed with the developed load test by PBC with HPs (Mercury) Loadrunner tooling based
on the Citrix ICA protocol over TCP/IP.
During development it became clear that this load test approach in combination with the SX.e
application (written in OpenEdge ABL) is not very robust. Lots of load test scriping effort is used
to overcome the lack of robustness in the protocol. This was a serious risk for the project.

2.5

Load Test description

The load test consists of one batch user, 6 user profiles in a mix of 20 users and Pick Ticket
printing. The following table gives some details. The mix of 20 users is scaled up during load
testing
Script

Mix ratio [%]

([#/20 user})

Transaction
duration [s]

Average
lines per
order

Remarks

OE Order
Entry

45% (9)

72

OEET, order size is between 2

and 25 lines.

OE Order
Shipping

10% (2)

42

N.A.

OEES

OE Order
Inquiry

5% (1)

180

OEIO

AP Costing

20% (4)

180

N.A.

APET

AR Cash
Receipts

15% (3)

72

N.A.

AREC

PO Order
Entry

5% (1)

72

POET, order size is between 2

and 30 lines

Application and DB testing

Average
lines per
order

5-3

Script

Mix ratio [%]

([#/20 user})

Transaction
duration [s]

Remarks

Login/Logout

1 batch user

variable

Frequency based on the amount

of users participating in the load
test

OE Pick
Ticket
Printing

1 report
process

120

Picks all entered orders of the 9

defined warehouses

The After-imaging archive script was run from cron every 2 minutes. The standard
aimage.archive script was modified to always save to the same filename (nxt.1). As AI extents
filled they were saved off to /ai/aisave/nxt.1 and marked as empty.

2.6

Other information

This load test results document contains high level information from the tests. The details are
saved on CD. The tests are numbered in the logbook, it contains information about the tests
and te result files. The project backup CD contains:

Logbook: \SXe 700 user Load Test Logbook.doc (PD#46663)

Design: \SX.e Load Test.pdf (PD#46664)
Test setup estimates: \Check Database.xls
Citrix Documentation: \Citrix-*
AIX profiling output of vmstat, iostat: \AIX\ResultsAIX.tar.gz
Performance Logs of nlbawpbc6 and nlbawpbc12: \PerfLogs\Perflogs.zip
Loadrunner raw results: \LR\ResultsA.zip (nlbawpbc4) and \LR\ResultsB.zip
(nlbawpbc10)
Load test sources (Rel1_1): \SX_Loadtest\...
LR analysis results as used in this document: \Analisys\...

3.Hardware setup

3.1

Hardware setup

A: nlbawpbc4
Loadrunner
Controller & Host
400 users

L: redant
Citrix License
SX.e Staging

1: nlbawpbc6
Citrix
240 SX.e Clients

2: nlbawpbc12
Citrix
220 SX.e Clients

S: nlbaupbc5
SX.e Application Server
Progress OpenEdge

Logical hardware layout of load test.

B: nlbawpbc10
Loadrunner
Controller & Host
300 users

3: nlbawpbc13
Citrix
240 SX.e Clients

3.2
System Name #

System
type

Involved hardware
CPU type/ CPU CPU
quantity
speed L1
(GHz) cache
(MB)

CPU
L2
cache
(MB)

CPU
L3
cache
(MB)

Internal Total HD #Spindles

memory cap.
(GB)
(GB)

137

111
200

1 Sata (7200)
4 SAN, Raid0

137

2 SCSI (15k)

31

nlbawpbc4

A Intel

64-bit
3.2
Intel
Xeon
7130
Dual,HT /4

Nlbawpbc6

2.66

nlbawpbc10
nlbawpbc12
nlbawpbc13

B Dell 1950
2
3

64-bit
Intel
Xeon
5150
Dual /2

Redant

Piii

1.13

nlbaupbc5

S IBM p570 Power5

Intel

IBM x330

0.5

1.5

36

20

4 Raid0
1
1

Network configuration is a mix of 100 Mb and 1Gb connections.

3.3 I/O configuration nlbaupbc5

Volume name

RAID level

Stripe
size (KB)

Function

Disk0

Raid-0

64

/db (nxt database)

/rd (SX.e and
OpenEdge)

Disk5

Disk6

/dump
/backup

/ai (transaction log)

/ (root)
/tmp
/var

/home
/opt

Application and DB testing

5-3

3.4 Database layout

Usage

Size
(MB)

Volume name

Type

SX.e Database

8000

/db

RAID 0

Logs

20

/rd/tmp

RAID 0

Sort files

N/A

/db/sort

RAID 0

Report Manager logs

After-imaging

/rd/tmp
1000

/ai

Database structure:
Extent

Size
(MB)

Quantity

Records/
Block

Blocks/
Cluster

Type

Before-Image

512

Fixed + 1 variable

Schema Area

64

64

Fixed + 1 variable

After-Image

100

10

Fixed

Default

128

10

128

64

Fixed + 1 variable

Default_idx

32

Fixed + 1 variable

Transient

50

10

128

64

Fixed + 1 variable

Transient_idx

32

Fixed + 1 variable

Addon

32

128

512

Fixed + 1 variable

Arett

32

64

512

Fixed + 1 variable

Icsw

32

16

512

Fixed + 1 variable

Icswu

32

16

512

Fixed + 1 variable

Oeeh

32

16

512

Fixed + 1 variable

Oeel

32

32

512

Fixed + 1 variable

Icsw_idx

32

Fixed + 1 variable

All Others (Data)

32

64

512

Fixed + 1 variable

All Others (Index)

32

Fixed + 1 variable

4.Software setup

4.1 Software versions + patches

Server

Software

Version

Patches

nlbawpbc4
nlbawpbc10

Operating System Windows

2003 Enterprise
Edition resp. 64
bit, 32 bit

SP2

Infor SX.e Load Test

Rel1_0

Rel1_1

Loadrunner

8.1

FP3

Infor BSW

1.6.1.3

Operating System Windows

2003 Enterprise
32 bit

SP2

Citrix Pressentation Server

4.0, Build 3790,

SP 2005.04

Hotfix rollup pack

PSE400W2K3R03

Load Runner Citrix Agent

pal_drv.sys

8.1.0.474
8.1.0.1079

FP3

SX.enterprise

5.1 GUI

Special fix for logon/logoff

Progress OpenEdge

10.0B.05.03

Operating system Windows

2000 Advanced

Citrix Metaframe Access Suite

Licensing

2..0.0

Microsoft SQL Server 2000

8.00.760

SX.enterprise

5.1 GUI

Progress OpenEdge

10.0B.05.03

Operating System AIX

5.3.0.30

Progress OpenEdge

10.0B.05.03

SX.enterprise

5.0 CHUI

nlbawpbc6
nlbawpbc12
nlbawpbc13

redant

nlbaupbc5

SP4

Special fix for logon/logoff

32-bit and 64-bit

4.2

Tuning

4.2.1 Citrix Servers

Added /PAE in the boot.ini file to access 8 GB of memory.
Disabled all disk and printer mappings in ICA protocol configuration
Configured windows 2003 server as non-file server with back ground applications.

4.2.2 Aix Server

Tuned the following AIX tunables:

VMO:

maxclient%
maxperm%
minperm%

20
20
5

IOO:
maxpgahead
32
maxrandwrt
128
numfsbufs
2K
sync_release_ilock
1

syncd 20

Other:

4.2.3

Progress OpenEdge

Tuned the following OpenEdge tunables:

Database:
Physical Database Name (-db): /db/nxt
Database Block Size (bytes): 8192
Direct I/O (-directio): Not Enabled
Number of Database Buffers (-B): 100000
Size of Lock Table (-L): 50000
Spin Lock Tries (-spin): 60000
Number of Semaphore Sets (-semsets): 3
Before-Image Block Size (bytes): 16384
Before-Image Cluster Size (Kbytes): 16384
Number of Before-Image Buffers (-bibufs): 30
Number of After-Image Buffers (-aibufs): 45
After-Image Block Size (bytes): 16384
After-Image Stall (-aistall): Enabled

Application and DB testing

Maximum Number of Servers (-Mn): 300

Minimum Clients Per Server (-Mi): 5
Maximum Clients Per Server (-Ma): 10
Maximum Number of Users (-n): 15000

Appserver:
maxClientInstance=800
maxSrvrInstance=800

5-3

5.Test Results

The tests focus on two topics. The sizing of the used citrix servers and the SX.e application
with database server. Prior to this a base line is measured with small load. This base line can
serve as a reference.

5.1 Response times with low load

This are the results of a test with only 140 users. It give a good impression of the impact of
higher volume tests on the response times of the various transactions.
The setup consisted of:

Load runner system nlbawpbc4

Citrix server nlbawpbc12
Application and Database server nlbaupbc5 with 2 CPUs and 12 GB of internal memory.
The OpenEdge version was the 32 bit version.

Application and DB testing

Analysis Summary
Scenario Name:

5-3

Period: 18/07/2007 11:58:06 - 18/07/2007 13:54:02

E:\SX.e-5.1\include\SXeTest140.lrs

Results in Session:

E:\temp\results\SXeTest140\SXeTest140.lrr

Duration:
Filters:
Statistics Summary

1 hour, 55 minutes and 56 seconds.

(4560 <= Scenario Elapsed Time) and (6060 >= Scenario Elapsed Time), (Include Think Time)

Maximum Running Vusers:

141

Transaction Summary
Transactions:

Total Passed:16,575 Total Failed:

19 Total Stopped:

Transaction Name
Minimum Average Maximum Std. Deviation90 Percent Pass
Fail Stop
Action_Transaction
31.093 80.872 907.082
68.386
179.918 2,346
10
0
AP-CostingHeader
0.406
0.419
2.844
0.16
0.422
230
0
0
AP-CostingLine
0.406
0.411
0.469
0.01
0.422
231
0
0
AR-CashReceipt
0.375
0.41
0.625
0.039
0.453
319
0
0
AR-CashReceipt2
0.382
0.423
0.611
0.071
0.611
8
0
0
LogOnOff
2.5
2.772
3.187
0.195
3.047
15
0
0
OE-InquiryHeader
0.391
0.411
0.484
0.013
0.422
60
0
0
OE-InquiryLineDetail
0.391
0.479
0.859
0.095
0.578
60
0
0
OE-InquiryLines
0.781
0.791
1.031
0.039
0.797
60
0
0
OE-InquirySearch
0.406
0.428
0.656
0.047
0.484
37
0
0
OE-OrderEnd
0.391
0.436
3.615
0.195
0.422 1,104
1
0
OE-OrderHeader
0.391
0.424
3.906
0.15
0.442 1,121
1
0
OE-OrderLine
0.391
0.417
3.844
0.099
0.424 5,564
0
0
OE-OrderRecalculate
0.766
0.787
1.234
0.022
0.801 1,106
0
0
OEOrderShippingShip
0.375
0.38
0.453
0.009
0.391
504
0
0
PO-OrderEnd
0.406
0.409
0.437
0.007
0.422
120
0
0
PO-OrderHeader
0.406
0.419
0.719
0.04
0.438
122
0
0
PO-OrderLine
0.391
0.411
0.766
0.022
0.422
798
0
0
PO-ProductLookup
0.766
0.793
4.109
0.165
0.797
407
0
0
Script_AP Costing
179.514 179.926 180.512
0.152
180.002
230
0
0
Script_AR Cash Receipts
71.506 71.961
72.513
0.204
72.251
321
5
0
Script_LogOnOff
101.59 102.022 102.498
0.329
102.486
15
0
0
Script_OE Order Inquiry
180.08 180.67 181.012
0.168
180.872
57
0
0
Script_OE Order Shipping
41.577 41.751
42.39
0.055
41.769
502
0
0
Script_OEOrderEntry
28.469
67.95
228.29
39.784
117.604 1,105
1
0
Script_PO Order Entry
31.093 81.991 364.735
53.694
149.324
120
0
0
vuser_end_Transaction
0
0.328
1.969
0.734
1.969
6
0
0
vuser_init_Transaction
39.55 49.632
91.66
18.069
91.66
7
1
0

Table 1 Transaction summery 'Low load'

5.2

Citrix server testing

This test focused on the maxium load that the the citrix server can handle. This only applies to
the citrix server as described before. This implies that the application / database server was not
allowed to be loaded more than 70%.
A number of tests have been applied with various user amounts (See logbook for more details).

6-4

Conclusions

The setup consisted of:

Load runner system nlbawpbc4

Citrix server nlbawpbc6
Application and Database server nlbaupbc5 with 2 CPUs and 12 GB of
internal memory. The OpenEdge version was the 32 bit version.

5.2.1 Observations
System under test behavior
Users

Value

Remark

280

9.0 GB

Memory overloaded hanging

processes?

240

6.4 GB

220

6.0 GB

200

5.8 GB

143

3.8 GB

220

60%

200

53%

Disk transfers
/sec

220

200 max

Raid 0

200

142 max

Disk (15k rpm)

Disk Avg. queue

length

220

0.000

Raid 0

200

0.066

Disk (15 rpm)

Network kB/sec

220

922

100 Mb

200

790

100 Mb

Memory usage

CPU (max)

Process memory usage

Class

Process

Private Bytes [MB]

Workingset [MB]

Load runner

Ctrxagent

1.6

5.0

Citrix / Terminal
Services

Csrss

1.1

3.1

Wfshell

2.6

6.3

winlogon

3.8

3.2

Prowin32

16.9

25.4

SX.e

These numbers are average across 10 processes. Note that the workingset includes shared
memory like DLLs etc.

Application and DB testing

Number of users over time

5-5

6-6

Conclusions

Analysis Summary
Scenario Name:

Period: 20/07/2007 12:32:28 - 20/07/2007 14:01:37

E:\SX.e-5.1\include\SXeTest700.lrs

Results in Session:

E:\temp\results\sxetest700-006\sxetest700-006.lrr

Duration:

1 hour, 29 minutes and 9 seconds.

Statistics Summary
Filters:
Maximum Running Vusers:

(3060 <= Scenario Elapsed Time) and (4680 >= Scenario Elapsed Time), (Include Think Time)
270

Transaction Summary
Transactions:

Total Passed:33,392 Total Failed:

33 Total Stopped:

Transaction Name
Action_Transaction
AP-CostingHeader
AP-CostingLine
AR-CashReceipt
AR-CashReceipt2
LogOnOff
OE-InquiryHeader
OE-InquiryLineDetail
OE-InquiryLines
OE-InquirySearch
OE-OrderEnd
OE-OrderHeader
OE-OrderLine
OE-OrderRecalculate
OEOrderShippingShip
PO-OrderEnd
PO-OrderHeader
PO-OrderLine
PO-ProductLookup
Script_AP Costing
Script_AR Cash Receipts
Script_LogOnOff
Script_OE Order Inquiry
Script_OE Order Shipping
Script_OEOrderEntry
Script_PO Order Entry
vuser_end_Transaction
vuser_init_Transaction

Minimum Average Maximum Std. Deviation 90 Percent Pass Fail Stop

19.516 78.118 348.259
50.768
179.863 4,933
19
0
0.406
0.409
0.425
0.002
0.413
498
0
0
0.403
0.406
0.438
0.003
0.409
498
0
0
0.377
0.408
1.304
0.097
0.414
771
0
0
0.395
0.395
0.395
0
0.395
1
0
0
2.522
2.993
3.662
0.276
3.447
61
0
0
0.394
0.41
0.443
0.006
0.412
126
0
0
0.392
0.459
0.821
0.109
0.6
126
0
0
0.781
0.784
0.8
0.003
0.786
126
0
0
0.392
0.431
1.423
0.122
0.445
69
0
0
0.389
0.458
1.61
0.219
0.424 2,161
0
0
0.391
0.413
1.485
0.05
0.413 2,156
3
0
0.389
0.41
1.44
0.023
0.414 10,777
2
0
0.766
0.785
1.769
0.023
0.786 2,167
0
0
0.375
0.384
0.899
0.02
0.394 1,086
0
0
0.391
0.414
1.485
0.071
0.411
232
0
0
0.403
0.407
0.442
0.005
0.409
230
0
0
0.389
0.406
0.466
0.005
0.41 1,622
0
0
0.763
0.781
0.813
0.003
0.786
797
1
0
179.505 179.907 180.506
0.181
179.896
500
1
0
71.508
71.93
72.511
0.212
72.449
777
1
0
19.516 20.057
20.483
0.295
20.469
60
0
1
180.021 180.608 181.004
0.147
180.723
126
0
0
41.718 41.748
42.247
0.027
41.769 1,087
0
0
28.525 69.949 242.692
41.408
123.893 2,164
3
0
26.827 86.667
348.26
56.426
154.163
231
1
0
0
0.757
1.784
0.874
1.784
7
0
0
52.663 62.477
67.401
6.939
67.401
3
2
0

Table 2 Transaction summery Citrix Server

5.3

Application and DB testing

In this test the focus is on the applcation and database server (nlbaupbc5). The goal was to go
up to 700 concurrent users. This implied that the loadrunner and citrix servers should not be
loaded too high. From the previous test we can conclude that we can safely go up to 220-240

Application and DB testing

5-7

users per citrix server. The load runner server nlbawpbc4 was fully loaded at 600 users, so we
added an other load runner server nlbawpbc10 simulating 300 users.
Starting up users is quite heavy compared to running the scripts, so we had to change the
scripts in a way that first all the users were getting in and than let the users start . This start was
randomized over a period of 5 min. The number of concurrent logons is about 10 per citrix
server.
Due to limitations in loadrunner it was not possible to drive the load generator systems (pbc4
and pbc10) from one console. This implies that two measurements were created A and B, but
because each load generator is equaly connection to each citrix server, the transaction
response times are equal.

5.3.1

300 users on application and database server

During 15 minutes 300 user were run from the nlbawpbc10.

Disks:
hdisk0
hdisk6
hdisk7

Data
Average
Average Max of Average
of
Max of Average Max of Average Max of of
Kb_rea of
Max of
tm_act tm_act of Kbps Kbps of tps
tps
Kb_read d
Kb_wrtn Kb_wrtn
6.93 18.60
1188 1753
106
197
2478
7028 33166 48808
0.32
1.70
5
33
1
4
12
96
135
920
6.65 13.40
702 3746
26
60
313
4620 20759 111740

Disk IO according iostat from nlbaupbc5

6-8

Conclusions

Sample
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
Average
Max

Mem:
Device
runn wai active
Mem: Pager Interru System Context User System Idle Wait
able ting pages
free list i/o
pts
calls
switches time time
time time
3
1 4349725
3947
0
1698 24572
5419
29
5
64
1
3
2 4350066
3969
0
1827 26994
5562
33
6
60
1
3
1 4350091
3921
0
1886 26753
5672
31
6
60
2
2
1 4350290
3985
0
1729 25027
5446
29
6
62
3
3
1 4350164
3880
0
1947 29507
5730
33
7
58
2
2
1 4350195
3865
0
1747 26316
5500
30
6
63
2
3
1 4350203
3883
0
1829 28075
5632
32
6
61
1
3
1 4350414
4232
0
1872 31431
5687
33
6
59
2
2
1 4350466
4066
0
1783 25706
5550
31
6
62
1
3
1 4350975
3884
0
1883 29155
5670
34
6
58
2
3
1 4350993
4096
0
1794 24732
5523
30
6
63
2
3
1 4350997
4297
0
1690 25652
5404
30
5
64
1
3
1 4351029
3939
0
1749 24798
5503
30
6
63
1
3
1 4351031
4057
0
1620 24433
5276
30
5
63
1
2
1 4351081
3848
0
1816 27583
5583
32
6
61
1
2
1 4351284 11061
0
1791 29026
5561
32
6
60
2
3
1 4351153
3913
0
1916 28143
5640
30
6
60
3
3
1 4351177
4016
0
1836 30446
5637
33
6
60
1
2
1 4351212
3842
0
1688 25053
5401
30
5
64
1
3
1 4351660
4308
0
1964 29816
5761
35
6
57
2
3
1 4351660
4067
0
1785 26075
5544
31
6
62
1
3
1 4351708
4004
0
1789 26469
5555
32
6
61
1
3
1 4351710
4092
0
1839 26913
5626
31
6
62
1
3
1 4351902
7917
0
1844 30124
5633
32
6
60
1
3
1 4351724
4072
0
1784 26425
5529
31
6
62
1
3
1 4351727
3920
0
1756 25191
5479
30
6
64
1
3
1 4351727
3848
0
1752 26458
5496
31
6
63
1
3
1 4351726
4066
0
1852 30424
5664
33
6
60
1
3
1 4351753
3881
0
1809 25715
5595
30
6
63
1
2
1 4351775
4014
0
1537 20727
5152
28
5
66
1
2.767
1 4351054
4363
0 1793.7 26924.6
5547.7 31.2
5.9 61.5 1.4
3
2 4351902 11061
0
1964 31431
5761
35
7
57
3

Vitual memory statistics according vmstat from nlbaupbc5.

Number of users
Overal period: 27/07/2007 14:34:02 - 27/07/2007 16:10:37.

Application and DB testing

Analysis Summary
Scenario Name:

5-9

Period: 27/07/2007 14:34:02 - 27/07/2007 16:10:37

E:\SX.e-5.1\include\SXeTest700b.lrs

Results in Session:

E:\temp\results\sxetest700b-013\sxetest700b-013.lrr

Duration:
Filters:
Statistics Summary

1 hour, 36 minutes and 35 seconds.

(1800 <= Scenario Elapsed Time) and (2700 >= Scenario Elapsed Time), (Include Think Time

Maximum Running Vusers:

300

Transaction Summary
Transactions:

Total Passed: 22,283 Total Failed:

Transaction Name
Action_Transaction
AP-CostingHeader
AP-CostingLine
AR-CashReceipt
AR-CashReceipt2
OE-InquiryHeader
OE-InquiryLineDetail
OE-InquiryLines
OE-InquirySearch
OE-OrderEnd
OE-OrderHeader
OE-OrderLine
OE-OrderRecalculate
OEOrderShippingShip
PO-OrderEnd
PO-OrderHeader
PO-OrderLine
PO-ProductLookup
Script_AP Costing
Script_AR Cash Receipts
Script_OE Order Inquiry
Script_OE Order Shipping
Script_OEOrderEntry
Script_PO Order Entry
vuser_end_Transaction
vuser_init_Transaction

Minimum Average Maximum Std. Deviation

90 PercentPass Fail Stop
26.812
80.464 1,543.86
75.428 179.887 2,740 527
0
0.402
0.407
0.412
0.001
0.412 300
0
0
0.403
0.41
1.674
0.073
0.412 300
0
0
0.376
0.383
0.387
0.002
0.387
36
0
0
0.375
0.383
0.406
0.003
0.393 523
0
0
0.404
0.407
0.41
0.001
0.41
75
0
0
0.38
0.416
0.511
0.043
0.491
74
0
0
0.78
0.782
0.786
0.001
0.785
74
0
0
0.391
0.407
0.41
0.003
0.41
37
0
0
0.388
0.431
1.658
0.157
0.414 1,525
0
0
0.389
0.407
1.391
0.026
0.412 1,524
0
0
0.388
0.408
1.689
0.03
0.413 7,699
1
0
0.764
0.783
1.862
0.039
0.786 1,524
0
0
0.375
0.379
0.391
0.001
0.384 648
0
0
0.392
0.407
0.411
0.002
0.411 159
0
0
0.39
0.406
0.41
0.002
0.41 158
0
0
0.388
0.405
0.412
0.005
0.412 1,075
1
0
0.766
0.781
0.788
0.001
0.784 540
0
0
179.51 179.936
180.51
0.205
180.5 300
0
0
71.557
71.948
72.449
0.147
71.984 558
0
0
180.23 180.561 180.714
0.111 180.637
74
0
0
41.717
41.748
41.768
0.011
41.768 649
0
0
37.932
68.705 231.456
40.389 119.027 1,525
2
0
26.812
84.325 345.696
55.641 153.291 158
1
0
0
1.127
1.71
0.797
1.71
3
0
0
29.107
47.291
60.852
11.508
60.852
5
0
0

Table 3 Transaction summery 300 users

532 Total Stopped: 0

6-10

Conclusions

5.3.2

700 users on application and database server

This test involved two loadrunner servers A (400 users) and B (300 users). Both servers did not
start at the same moment so there is time offset differences between the data in the load runner
graphs.
Data

Disks:
hdisk0
hdisk6
hdisk7

Average
Average of Max of
Average Max of Average Max of of
Max of
%tm_act %tm_act of Kbps Kbps of tps
tps
Kb_read Kb_read
16.7
39.2
3645 5133
335
490
3439
13712
0.4
1.7
6
36
1
5
27
292
17.3
26.0
1805 7472
64
116
114
1592

Average
of
Kb_wrtn
105898
152
54035

Max of
Kb_wrtn
146912
1048
224128

Table 4 Disk IO according iostat from nlbaupbc5

Note: logging all users off at the same moment saturates the disks 0 and 7 for some time.

Application and DB testing

5-11

Mem:
Mem:
Device
runn wait active
free
Page Interru System Context User Syste Idle Wait
Sample able ing pages
list
r i/o pts
calls
switches time m time time time
1
5
1 4360963 19793
0
3351 49449
7808
58
9
32
2
2
7
1 4361146
4090
0
3469 53179
7757
63
9
26
2
4
7
1 4361519
3873
0
3477 51972
7877
63
9
27
1
5
6
1 4361769
4246
0
3556 52967
8006
59
9
29
3
6
7
1 4361686
3917
0
3627 53031
8046
62
9
26
2
7
9
1 4361733
4006
0
3625 55659
8027
64
9
25
2
8
7
1 4361986
3906
0
3554 52798
7942
62
9
27
2
9
7
1 4362779 22001
0
3545 53046
7950
62
9
27
2
11
7
1 4363257
3894
0
3492 54596
7903
64
9
26
1
12
7
1 4363197
3906
0
3696 55814
8023
64
9
25
2
13
7
1 4363470
4327
0
3481 55371
7844
60
9
28
2
14
6
1 4363954
3928
0
3534 52865
7889
65
9
25
1
16
7
1 4368933
3911
0
3562 55235
8029
65
9
25
1
17
6
1 4369062
4406
0
3443 51797
7821
60
9
29
1
18
8
1 4369080
3870
0
3589 52350
7944
62
9
27
2
19
7
1 4369091
3944
0
3422 53081
7787
62
9
28
2
21
8
1 4374713
4586
0
3499 58288
7921
69
9
20
2
22
8
1 4375170
4269
0
3548 56684
7740
72
9
18
1
23
9
1 4375231
4740
0
3507 57586
7583
72
9
18
1
24
8
1 4375252
3875
0
3531 55632
7697
73
9
17
2
25
9
1 4375347
4366
0
3519 58562
7819
72
9
18
1
27
8
1 4375906
4043
0
3490 54558
7621
71
8
19
2
28
7
1 4376049
3922
0
3443 51068
7454
72
8
18
2
29 10
1 4376566
4527
0
3614 62354
8041
72
9
17
2
30
8
1 4376730
3938
0
3408 53054
7511
71
8
18
2
32
9
1 4376955
3963
0
3605 58565
7728
75
9
15
2
33
7
1 4377224
5516
0
3482 62277
7606
71
10
17
2
34
8
1 4377399
4254
0
3448 53702
7743
70
8
20
1
35
8
1 4377409
4078
0
3492 55265
7588
73
8
18
1
36
8
1 4377415
4259
0
3607 57871
7816
73
9
17
1
37
7
1 4377431
4590
0
3596 56361
7700
71
9
18
3
39
8
1 4378267
4804
0
3512 56260
8031
67
9
23
1
40
6
1 4378281
3865
0
3554 50270
7877
61
9
29
2
41
8
1 4378527
4289
0
3714 58200
7943
67
10
20
3
42
9
1 4379117
3993
0
3662 58788
7680
74
9
16
2
43
9
1 4379282
5184
0
3440 55890
7821
65
9
25
1
44
6
1 4379288
3844
0
3599 53522
8023
63
9
26
2
46
9
1 4379600
3856
0
3522 56441
7890
72
9
18
1
47
6
1 4379886
4313
0
3503 50345
7860
63
9
27
2
48
6
1 4379920
3935
0
3438 50910
7849
62
8
28
1
49
7
1 4379933
4514
0
3298 52905
7453
66
9
23
2
50
8
1 4379939
3904
0
3330 56210
7693
68
8
21
2
Average
7
1 4364916
5912
0
3543 54115
7937
63
9
26
2
Max
9
1 4374713 22001
0
3713 60331
8116
72
9
32
3

Virtual memory statistics according to vmstat from nlbaupbc5 (Some lines are hided to make it
fit).

6-12

Conclusions

Number of user on nlbawpbc4 (A)

Number of users on nlbawpbc10 (B)

Application and DB testing

Analysis Summary

Period: 27/07/2007 14:29:44 - 27/07/2007 16:19:35

Scenario Name:

E:\SX.e-5.1\include\SXeTest700a.lrs

Results in Session:

E:\temp\results\sxetest700a-0013\sxetest700a-0013.lrr

Duration:
Filters:
Statistics Summary

1 hour, 49 minutes and 51 seconds.

(3900 <= Scenario Elapsed Time) and (5400 >= Scenario Elapsed Time), (Include Think Time)

Maximum Running Vusers:

401 / 700

Transaction Summary
Transactions:

Total Passed:45,501 Total Failed:

1,312 Total Stopped:

Transaction Name
Minimum Average Maximum Std. Deviation
90 Percent Pass
Fail Stop
Action_Transaction
26.823 80.666 432.133
55.845
179.89 5,573 1,288
0
AP-CostingHeader
0.402
0.409
0.532
0.006
0.414
668
0
0
AP-CostingLine
0.392
0.421
2.719
0.139
0.412
677
0
0
AR-CashReceipt
0.391
0.391
0.391
0
0.391
2
0
0
AR-CashReceipt2
0.375
0.398
0.603
0.016
0.414 1,250
0
0
LogOnOff
4.178
5.218
7.896
0.716
5.849
42
0
0
OE-InquiryHeader
0.405
0.41
0.486
0.007
0.413
167
0
0
OE-InquiryLineDetail
0.389
0.737
2.599
0.429
1.424
169
0
0
OE-InquiryLines
0.778
0.786
0.851
0.008
0.791
167
0
0
OE-InquirySearch
0.393
0.445
1.424
0.18
0.429
88
0
0
OE-OrderEnd
0.388
0.792
5.642
0.643
1.535 2,956
0
0
OE-OrderHeader
0.391
0.47
2.802
0.261
0.433 2,984
0
0
OE-OrderLine
0.385
0.448
2.783
0.223
0.414 15,249
5
0
OE-OrderRecalculate
0.76
1.079
6.628
0.766
1.975 2,959
1
0
OEOrderShippingShip
0.375
0.384
0.837
0.016
0.394 1,433
0
0
PO-OrderEnd
0.392
0.452
1.941
0.218
0.413
336
1
0
PO-OrderHeader
0.401
0.432
1.673
0.159
0.424
339
0
0
PO-OrderLine
0.388
0.432
2.343
0.182
0.414 2,402
2
0
PO-ProductLookup
0.764
0.782
0.977
0.009
0.786 1,163
0
0
Script_AP Costing
179.505 179.918 180.512
0.179 180.113
666
3
0
Script_AR Cash Receipts
47.965 72.047
72.51
0.744
72.414 1,247
2
0
Script_LogOnOff
35.606
36.06
36.478
0.277
36.424
41
0
0
Script_OE Order Inquiry
180.022 180.479 180.975
0.253 180.725
162
0
0
Script_OE Order Shipping
41.687 41.757
42.467
0.047
41.769 1,439
0
0
Script_OEOrderEntry
37.926 71.719 255.343
42.933 126.849 2,956
7
0
Script_PO Order Entry
26.823 85.341 338.104
53.774 149.165
335
3
0
vuser_end_Transaction
0
1.487
11.741
2.867
1.803
15
0
0
vuser_init_Transaction
31.543 48.894
64.982
10.116
63.857
16
0
0

Table 5 Transaction summery 700 users A

Analysis Summary
Scenario Name:
Results in Session:
Duration:
Filters:
Statistics Summary
Maximum Running Vusers:

Period: 27/07/2007 14:34:02 - 27/07/2007 16:10:37

E:\SX.e-5.1\include\SXeTest700b.lrs
E:\temp\results\sxetest700b-013\sxetest700b-013.lrr
1 hour, 36 minutes and 35 seconds.
(3450 <= Scenario Elapsed Time) and (4950 >= Scenario Elapsed Time), (Include Think Time)
300 / 700

Transaction Summary
Transactions:

Total Passed: 34,029 Total Failed:

Transaction Name
Action_Transaction
AP-CostingHeader
AP-CostingLine
AR-CashReceipt
AR-CashReceipt2
OE-InquiryHeader
OE-InquiryLineDetail
OE-InquiryLines
OE-InquirySearch
OE-OrderEnd
OE-OrderHeader
OE-OrderLine
OE-OrderRecalculate
OEOrderShippingShip
PO-OrderEnd
PO-OrderHeader
PO-OrderLine
PO-ProductLookup
Script_AP Costing
Script_AR Cash Receipts
Script_OE Order Inquiry
Script_OE Order Shipping
Script_OEOrderEntry
Script_PO Order Entry
vuser_end_Transaction
vuser_init_Transaction

Minimum Average Maximum Std. Deviation

90 PercentPass Fail Stop
26.81
80.076 343.289
54.369 179.889 4,196 967
0
0.4
0.421
2.297
0.141
0.411
493
0
0
0.39
0.416
1.686
0.108
0.412
493
0
0
0.376
0.383
0.391
0.004
0.391
13
0
0
0.372
0.383
0.408
0.003
0.393
923
0
0
0.404
0.407
0.423
0.002
0.411
124
0
0
0.38
0.575
2.002
0.281
0.941
124
0
0
0.778
0.782
0.787
0.001
0.786
123
0
0
0.391
0.424
1.407
0.13
0.411
58
0
0
0.385
0.795
5.485
0.643
1.55 2,255
1
0
0.385
0.475
2.517
0.282
0.414 2,255
0
0
0.38
0.449
3.094
0.233
0.414 11,349
5
0
0.763
1.083
6.204
0.734
2.018 2,255
0
0
0.373
0.379
0.766
0.012
0.383 1,071
2
0
0.391
0.436
1.594
0.173
0.411
256
1
0
0.39
0.47
2.468
0.279
0.41
260
0
0
0.385
0.427
2.328
0.173
0.413 1,728
2
0
0.765
0.782
1.748
0.033
0.786
872
1
0
179.511 179.935 180.512
0.199 180.479
498
0
0
47.202
71.89
72.466
1.141
71.984
938
3
0
180.042 180.519
181.01
0.214 180.702
123
1
0
41.515
41.749
42.152
0.027
41.768 1,074
1
0
37.928
70.795 253.378
41.264 121.575 2,255
9
0
26.811
82.873 343.289
52.143 141.529
256
5
0
0
2.49
11.708
4.057
11.68
19
0
0
28.299
41.589
63.625
10.324
60.525
18
0
0

998 Total Stopped:

Table 6 Transaction summery 700 users B

5-13

6-14

Conclusions

5.4

Loadfactor model testing

To be able to translate these figures into sizing data we need to know the load per user profile.
To accomplish this test have been executed for each user profile with 100 users concurrently.
The relative results are mentioned below The results of the OE Entry are split into the actual
Order Entry and the OE Processing pick tickets CPU loads.
CitrixServer
Profile
Avg CPU
OE Order Entry
26%
OE Order Shipping
2.2%
OE Order Inquiry
6.4%
AP Costing
2.2%
AR Cash Receipts
4.2%
Pick Tickets
PO Order Entry
29.0%

Application+DB Server
Citrix+ Application+DB Server
Relative Load
Relative Load
Relative Load
Load
Factor Avg CPU Load
Factor Avg CPU Load
Factor
37%
12
22%
49%
16
48%
42%
14
3%
1
1.6%
4%
1
4%
3%
1
9%
3
2.2%
5%
2
9%
8%
3
3%
1
1.3%
3%
1
4%
3%
1
6%
2
2.0%
5%
2
6%
5%
2
6.0%
14%
5
6%
5%
2
41%
14
9.4%
21%
7
38%
34%
11

Based on this the load factor model for the benchmark mix can be calculated for Application+
DB server:
Benchmark Load
`
mix [#]
mix
OE Order Entry
9
144
OE Order Shipping
2
2
OE Order Inquiry
1
2
AP Costing
4
4
AR Cash Receipts
3
6
Pick Tickets
1
5
PO Order Entry
1
7

Total load [#/20 users]

170
Average load [#/user]
8.5

6.Conclusions

6.1

Responsiveness of SX.e

When one compare the average response times of the transaction in the tables Table 1
(page3), Table 2 (page 6), Table 3 (page 9), Table 5 (page 13) and Table 6 (page 13), one can
see that most response times are below one second and hardly change. when the number of
users increases. Three transactions OE-InqiryLineDetail, OE-OrderEnd and
OE-OrderRecalculate (marked in the tables with orange) are sensitive for more load. One
can see a jump of 40% to 80% on the average response times between 300 user (Table 3), and
700 users (Table 5). The 90% is even higher (between 100% and 270%). This can be in
indication that there is some saturation in the system. A likely candidate is IO (See table Table
4 (page 10)). The max IO (measured over 30 seconds) is 490 per second that is close to the
max 6 * 100 IOs per second, per disk.

6.2

Resource consuption per user

The information from Chapter 5 Test Results is used to construct the following high level
estimates for resource consuption on the citrix server and de application/database server
Citrix Server
Resource
CPU (100%=1 Core)

SX.e Client

SX.e Application / DB server

Citrix Server

1.1 %

0%

17 MB

7.5 MB

IO

1 req/sec

Network

4 kB/sec

Memory

OpenEdge

SX.e Application

6.3

Transactions per hour, day for 700 users

Based on the measurements and the presented configuration one can present the number of
transactions per hour and day (24 hours) for the major transactions. This is inclusive OE Order
picking reporting for all entered orders.
AP Costing
AR Cash Receipt
OE Inquiry
OE Order Entry
OE shipping
PO Order Entry
Logon/Logoff

Transactions Transactions Lines [# per Lines [# per

[# per hour] [# per day]
hour]
day]
2,786
66,874
5,251
126,029
698
16,762
1,397
33,523
12,574
301,766
62,868
1,508,832
6,010
144,230
6,010
144,230
1,438
34,502
10,063
241,517
101
2,419

6.4 Load test execution

The load test execution was a process with lots of minor problems to tackle. Next to the normal
tuning effort problems had been observed with the stability of the load test execution tooling. A
number of those issues are blocking for load testing with higher number of users. Blocking
issues are:

Load runners citrix agent is only available on 32 bit windows and not on 64 bit windows.
This will result in large number of citrix servers running SX.e client due to memory
limitations of 32 bit windows
Load runners controller is capable to connect to remote load generator systems but the
error handling in the load runner scripts is not stable. This is due to a bug, but it is not clear
when this is fixed.
The problem of loosing keystrokes makes the script error prone. In the executed runs we
lost one user per 1 minute, however running into this problems goes along with time-out
delays in the scripts effecting the amount of transactions. These problems seem to emerge
when the load gets higher. This makes it difficult to stress test the systems up to the limit.

The overall conclusion is that the current laod test implementation is not good enough to go to
high end load testing; more than 1000 users.