Skip to content
GitLab
Commit 49b1d5b5 authored by Florent D'Halluin's avatar Florent D'Halluin
Browse files

Update cbs. 

CBS version: 17a18a174536d6b05c5364dd5f96f420b5ca75ef.
See cbs/Changelog.
parent daee8c11
Hide whitespace changes
Inline Side-by-side
ChangeLog
View file @ 49b1d5b5
2009-06-14 d-halluin <d-halluin@lrde.epita.fr>
Update cbs.
CBS version: 17a18a174536d6b05c5364dd5f96f420b5ca75ef.
See cbs/Changelog.
2009-06-05 Alexandre Duret-Lutz <adl@lrde.epita.fr>
 
* bootstrap: Update to mention cbs/ instead of argp/.
......
cbs/.gitignore
View file @ 49b1d5b5
......@@ -18,3 +18,8 @@ m4/ltoptions.m4
m4/ltsugar.m4
m4/ltversion.m4
m4/lt~obsolete.m4
*.data
*.eps
*.plot
_build
*.tar.gz
cbs/ChangeLog
View file @ 49b1d5b5
2009-06-14 d-halluin <d-halluin@lrde.epita.fr>
Libbench: add total number of timer measures.
* include/cbs/bench/timer.hh,
* include/cbs/bench/timer.hxx,
* lib/timer.cc: Implement.
* lib/bench.cc: Add in results.
2009-06-09 d-halluin <d-halluin@lrde.epita.fr>
Libbench: add system infos.
* lib/bench.cc: Add info.
2009-06-08 d-halluin <d-halluin@lrde.epita.fr>
Update demos.
* demo/comparison/main.cc,
* demo/generate-plot-files.sh,
* demo/memplot/main.cc: Update.
2009-06-08 d-halluin <d-halluin@lrde.epita.fr>
Update memplot: Add gnuplot & text output, fix bugs.
* include/cbs/bench/bench.hh: Add gnuplot format.
* include/cbs/bench/memplot.hh: Fix time probe.
* lib/bench.cc: Add gnuplot format support.
* lib/memplot.cc: Add new output, fix time probe.
* lib/timer.cc: Add missing \n.
2009-06-08 d-halluin <d-halluin@lrde.epita.fr>
Update documentation.
* README,
* README_LIBBENCH,
* TODO: Update.
2009-06-08 d-halluin <d-halluin@lrde.epita.fr>
Update .gitignore.
* .gitignore: Update.
2009-06-02 d-halluin <d-halluin@lrde.epita.fr>
Add XSD file draft.
* cbs.xsd: Add (incomplete).
2009-06-02 d-halluin <d-halluin@lrde.epita.fr>
Update TODO.
* TODO: Update.
2009-05-02 d-halluin <d-halluin@lrde.epita.fr>
Update stats script file.
* stats.sh: Update.
2009-05-02 d-halluin <d-halluin@lrde.epita.fr>
Add stats script file.
* stats.sh: New.
2009-05-01 d-halluin <d-halluin@lrde.epita.fr>
Update helper scripts.
(All .plot.in files): Add macro from machine and machine shot
name.
* demo/generate-plot-files.sh: Update.
* demo/vcsn/get-data.sh: Update.
2009-05-01 d-halluin <d-halluin@lrde.epita.fr>
Tweak & document demos.
* demo/memplot-bench/README,
* demo/timer-bench/README: More infos.
* demo/timer-bench/main.cc: Tweak.
* demo/vcsn/vcsn-quotient-fix.data,
* demo/vcsn/vcsn-quotient-orig.data: Remove (generated).
2009-05-01 d-halluin <d-halluin@lrde.epita.fr>
Add basic machine selection script for plots.
* demo/generate-plot-files.sh: New.
* README_LIBBENCH: Update.
Rename plot files in demo/.
* memplot-bench/results-memory.plot: Renamed as:
memplot-bench/results-memory.plot.in.
* memplot-bench/results-time.plot: Renamed as:
memplot-bench/results-time.plot.in.
* timer-bench/timer-many-tasks-memory.plot: Renamed as:
timer-bench/timer-many-tasks-memory.plot.in.
* timer-bench/timer-many-tasks-time.plot: Renamed as:
timer-bench/timer-many-tasks-time.plot.in.
* timer-bench/timer-recursion-memory.plot: Renamed as:
timer-bench/timer-recursion-memory.plot.in.
* timer-bench/timer-recursion-time.plot: Renamed as:
timer-bench/timer-recursion-time.plot.in.
* timer-bench/timer-scoped-memory.plot: Renamed as:
timer-bench/timer-scoped-memory.plot.in.
* timer-bench/timer-scoped-time.plot: Renamed as:
timer-bench/timer-scoped-time.plot.in.
* vcsn/vcsn-composition-normalized-memory.plot: Renamed as:
vcsn/vcsn-composition-normalized-memory.plot.in.
* vcsn/vcsn-composition-normalized.plot: Renamed as:
vcsn/vcsn-composition-normalized.plot.in.
* vcsn/vcsn-composition-rw-memory.plot: Renamed as:
vcsn/vcsn-composition-rw-memory.plot.in.
* vcsn/vcsn-composition-rw.plot: Renamed as:
vcsn/vcsn-composition-rw.plot.in.
* vcsn/vcsn-determinize-memory.plot: Renamed as:
vcsn/vcsn-determinize-memory.plot.in.
* vcsn/vcsn-determinize.plot: Renamed as:
vcsn/vcsn-determinize.plot.in.
* vcsn/vcsn-quotient-bool-memory.plot: Renamed as:
vcsn/vcsn-quotient-bool-memory.plot.in.
* vcsn/vcsn-quotient-bool.plot: Renamed as:
vcsn/vcsn-qu.ent-bool.plot.in: .
* vcsn/vcsn-quotient-z-memory.plot: Renamed as:
vcsn/vcsn-quotient-z-memory.plot.in.
* vcsn/vcsn-quotient-z.plot: Renamed as:
vcsn/vcsn-quo.nt-z.plot.in.
* vcsn/v.-quotient.plot: Renamed as:
vcsn/vcsn-quotient.plot.in.
2009-05-01 d-halluin <d-halluin@lrde.epita.fr>
Add/tweak plot examples & helper scripts.
Note: allow changing the current machine easily.
* demo/vcsn/display-figures.sh,
* demo/vcsn/generate-figures.sh,
* demo/vcsn/get-data.sh: Do that.
* demo/memplot-bench/results-memory.plot,
* demo/memplot-bench/results-time.plot,
* demo/timer-bench/timer-many-tasks-memory.plot,
* demo/timer-bench/timer-many-tasks-time.plot,
* demo/timer-bench/timer-recursion-memory.plot,
* demo/timer-bench/timer-recursion-time.plot,
* demo/timer-bench/timer-scoped-memory.plot,
* demo/timer-bench/timer-scoped-time.plot: Update.
* demo/vcsn/vcsn-composition-normalized-memory.plot,
* demo/vcsn/vcsn-composition-normalized.plot,
* demo/vcsn/vcsn-composition-rw-memory.plot,
* demo/vcsn/vcsn-composition-rw.plot,
* demo/vcsn/vcsn-determinize-memory.plot,
* demo/vcsn/vcsn-determinize.plot,
* demo/vcsn/vcsn-quotient-bool-memory.plot,
* demo/vcsn/vcsn-quotient-bool.plot,
* demo/vcsn/vcsn-quotient-z-memory.plot,
* demo/vcsn/vcsn-quotient-z.plot,
* demo/vcsn/vcsn-quotient.plot: New.
* demo/vcsn/vcsn-quotient-fix-1.data: Remove.
* demo/vcsn/vcsn-quotient-fix.data,
* demo/vcsn/vcsn-quotient-orig.data: Update.
2009-04-29 d-halluin <d-halluin@lrde.epita.fr>
Add bench makefile target.
* Makefile.am,
* demo/Makefile.am,
* demo/memplot-bench/Makefile.am,
* demo/timer-bench/Makefile.am: Do that.
2009-04-26 d-halluin <d-halluin@lrde.epita.fr>
(Memplot) Add relative memory usage, update demos.
* include/cbs/bench/memplot.hh,
* lib/bench.cc,
* lib/memplot.cc: Add relative_max.
* README_LIBBENCH: More infos.
* demo/memplot-bench/generate-figures.sh: New.
* demo/memplot-bench/get-results.sh: Delete.
* demo/timer-bench/generate-figures.sh: New.
* demo/timer-bench/get-results.sh: Delete.
* demo/memplot-bench/main.cc,
* demo/timer-bench/main.cc: Fork() so that measures
are independent.
* demo/memplot-bench/results-memory.plot,
* demo/memplot-bench/results-time.plot,
* demo/timer-bench/timer-many-tasks-memory.plot,
* demo/timer-bench/timer-many-tasks-time.plot,
* demo/timer-bench/timer-many-tasks.plot,
* demo/timer-bench/timer-recursion-memory.plot,
* demo/timer-bench/timer-recursion-time.plot,
* demo/timer-bench/timer-scoped-memory.plot,
* demo/timer-bench/timer-scoped-time.plot,
* demo/timer-bench/timer-scoped.plot,
* demo/vcsn/vcsn-quotient.plot: Update.
2009-04-26 d-halluin <d-halluin@lrde.epita.fr>
Update README_LIBBENCH figures.
......
cbs/Makefile.am
View file @ 49b1d5b5
......@@ -17,3 +17,6 @@ include $(top_srcdir)/cbs.mk
ACLOCAL_AMFLAGS = -I m4
SUBDIRS = lib demo include
bench:
cd demo && $(MAKE) $(AM_MAKEFLAGS) bench
cbs/README
View file @ 49b1d5b5
......@@ -6,3 +6,4 @@ tools for the Vaucanson library.
CBS contains:
* libbench: measure performance of equiped C++ code.
* plot helpers: extract results from xml files in gnuplot format.
cbs/README_LIBBENCH
View file @ 49b1d5b5
......@@ -27,9 +27,23 @@ Memory:
-- Benchmarking on another machine --
Adapt and use demo/generate-plot-files.sh to generate proper plot files
for your system.
Use the timer-bench and memplot-bench demos to compute those figures
on your system.
Note on the memory measures: For each module, although all the measures
are written in a single source file, the test process uses fork()
to make sure measures are independent.
-- Vaucanson Benchmark --
Some helper scripts are provided to use Vaucanson benchmark results:
1) Adapt and run demo/generate-plot-files.sh
2) Look in demo/vcsn, adapt and use .sh files.
## Typical usage ##
......@@ -40,6 +54,9 @@ ________________________________________________________
#include <cbs/bench/bench_macros.hh>
void do_some_tests();
void work();
int main()
{
BENCH_START("My benchmark", "My description");
......@@ -78,6 +95,10 @@ void do_some_tests()
// tests ends here.
}
void work()
{
}
________________________________________________________
Note:
......
cbs/TODO
View file @ 49b1d5b5
To do:
- Add more system info (e.g. hostname)
- Fix date in xml output (\n)
- Write proper XSD file, adapt XML output.
~ Allow bench loops:
CBS works with vcsn's global bencher.
......@@ -8,13 +11,15 @@ To do:
~ Support Windows & Mac:
Write windows & mac versions of the timing primitives.
vcsn:
- Equip bench:
Done, run make bench on a powerful machine
~ Consider changing memory probe to sbrk(0) again.
vcsn:
- Gather xml files:
Keep the directories as they are in make bench?
- Plot algorithms together between versions (bars)
Use one version as reference?
- Generate html:
Probably not necessary.
Extraction tool in bin/plot.pl
cbs/cbs.xsd 0 → 100644
View file @ 49b1d5b5
<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">
<xs:element name="bench">
<xs:complexType>
<xs:sequence>
<xs:element ref="name"/>
<xs:element ref="date"/>
<xs:element ref="time"/>
<xs:element ref="description"/>
<xs:element ref="parameters"/>
<xs:element ref="results"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="name" type="xs:string"/>
<xs:element name="date" type="xs:string"/>
<xs:element name="time" type="xs:integer"/>
<xs:element name="description" type="xs:string"/>
<xs:element name="parameters">
<xs:complexType>
<xs:sequence>
<xs:element ref="parameter"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="parameter">
<xs:complexType>
<xs:attribute name="name" use="required" type="xs:NCName"/>
<xs:attribute name="value" use="required" type="xs:string"/>
</xs:complexType>
</xs:element>
<xs:element name="results">
<xs:complexType>
<xs:sequence>
<xs:element maxOccurs="unbounded" ref="result"/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="result">
<xs:complexType>
<xs:attribute name="name" use="required"/>
<xs:attribute name="value" use="required" type="xs:string"/>
</xs:complexType>
</xs:element>
</xs:schema>
cbs/demo/Makefile.am
View file @ 49b1d5b5
......@@ -15,3 +15,9 @@
include $(top_srcdir)/cbs.mk
SUBDIRS = timer bench timer-bench memplot comparison memplot-bench
bench: bench.do
bench.do:
cd timer-bench && $(MAKE) $(AM_MAKEFLAGS) bench
cd memplot-bench && $(MAKE) $(AM_MAKEFLAGS) bench
cbs/demo/comparison/main.cc
View file @ 49b1d5b5
......@@ -69,6 +69,12 @@ int main(int argc, char** argv)
BENCH_SAVE(name + ".xml", bench::Options());
BENCH_SAVE(name + ".dot", bench::Options(bench::Options::VE_FULL,
bench::Options::FO_DOT));
BENCH_SAVE(name + "_simple.dot", bench::Options(bench::Options::VE_MINIMAL,
bench::Options::FO_DOT));
BENCH_SAVE(name + ".plot", bench::Options(bench::Options::VE_NORMAL,
bench::Options::FO_GNUPLOT,
bench::Options::CO_MEMPLOT));
}
void parent(int n)
......
cbs/demo/generate-plot-files.sh 0 → 100755
View file @ 49b1d5b5
#!/bin/bash
MACHINE="2.33GHz Intel Core 2 Quad Q8200, 4GB RAM"
MEXT="q8200"
for file in `find . -name "*.plot.in"`
do
LEN=`expr length $file`
let LEN="LEN-3"
out=`expr substr $file 1 $LEN`
sed "s/@MACHINE@/${MACHINE}/" $file > $out
sed -i "s/@MEXT@/${MEXT}/" $out
done
cbs/demo/memplot-bench/Makefile.am
View file @ 49b1d5b5
......@@ -18,3 +18,6 @@ noinst_PROGRAMS = demo-memplot-bench
demo_memplot_bench_SOURCES = main.cc
demo_memplot_bench_LDADD = $(top_builddir)/lib/libbench.la
demo_memplot_bench_CPPFLAGS = $(AM_CPPFLAGS)
bench: all
./demo-memplot-bench
cbs/demo/memplot-bench/README 0 → 100644
View file @ 49b1d5b5
1. Execute benches in build dir:
make && ./demo-memplot-bench
2. Generate figures in src dir:
./generate_figures.sh.
cbs/demo/memplot-bench/generate-figures.sh 0 → 100755
View file @ 49b1d5b5
#!/bin/sh
../../bin/plot.pl -a ../../_build/demo/memplot-bench/memplot_*.xml > memplot-bench.data
gnuplot results-memory.plot
gnuplot results-time.plot
cbs/demo/memplot-bench/get-results.sh deleted 100755 → 0
View file @ daee8c11
#!/bin/sh
../../../bin/plot.pl -a memplot_*.xml > memplot-bench.data
cbs/demo/memplot-bench/main.cc
View file @ 49b1d5b5
......@@ -14,17 +14,41 @@
//
#include <iostream>
#include <sys/types.h>
#include <sys/wait.h>
#include <cbs/bench/timer.hh>
#include <cbs/bench/bench_macros.hh>
using namespace timer;
static pid_t fork_in()
{
pid_t forked = fork();
if (forked < 0)
{
std::cerr << "Warning: failed to fork, running in parent." << std::endl
<< "Memory usage is most likely inaccurate." << std::endl;
}
return forked;
}
int main()
{
int n;
for (n = 10000; n < 500000; n *= 2)
{
// Do nasty stuff in another process so that the memory usage is accurate
pid_t forked = fork_in();
if (forked > 0)
{
// Parent waits
waitpid(forked, NULL, 0);
continue;
}
BENCH_START("memplot", "Empty tasks.");
for (int i = 0; i < n; ++i)
......@@ -54,6 +78,9 @@ int main()
BENCH_SAVE(name + ".dot", bench::Options(bench::Options::VE_MINIMAL,
bench::Options::FO_DOT));
}
if (forked == 0)
exit(0); // kill the child;
}
}
cbs/demo/memplot-bench/results-memory.plot deleted 100644 → 0
View file @ daee8c11
set title "Memory consumption of CBS Memplot (2GHz Intel Centrino, 1GB RAM)"
set xlabel "Number of measures"
set ylabel "Memory usage (MB)"
set key top left
plot 'memplot-bench.data' using ($1):($2/1024/1024) t "Memory consumption" smooth unique w linespoints pt 5 lc 1 lw 2
cbs/demo/memplot-bench/results-time.plot deleted 100644 → 0
View file @ daee8c11
set title "Time consumption of CBS Memplot (2GHz Intel Centrino, 1GB RAM)"
set xlabel "Number of measures"
set ylabel "Time taken (ms)"
set key top left
plot 'memplot-bench.data' using ($1):($3) t "CPU time" smooth unique w linespoints pt 5 lc 1 lw 2, \
'memplot-bench.data' using ($1):($4) t "user" smooth unique w linespoints pt 6 lc 1, \
'memplot-bench.data' using ($1):($5) t "system" smooth unique w linespoints pt 3 lc 1
cbs/demo/memplot/main.cc
View file @ 49b1d5b5
......@@ -20,13 +20,15 @@
int main()
{
for (int n = 1; n <= 4; ++n)
{
BENCH_START("Bench demo", "A simple benchmark with a single parameter");
BENCH_START("Bench demo", "A simple benchmark with a single parameter");
int maxn = 6;
std::cout << "[" << n << "] Bench for n=" << n << "..." << std::endl;
std::cout << "Benchmarking (n = " << maxn << ")..." << std::endl;
int n = 1;
for (; n <= maxn; ++n)
{
std::list<int> ilist;
std::list<std::string> slist;
......@@ -53,23 +55,26 @@ int main()
slist.clear();
BENCH_MEMPLOT("slist cleared");
BENCH_STOP();
}
BENCH_PARAMETER("n", (long) n);
BENCH_STOP();
std::stringstream name_str;
name_str << "benchmark_" << n;
std::string name = name_str.str();
BENCH_PARAMETER("n", (long) maxn);
BENCH_PRINT(bench::Options(bench::Options::VE_NORMAL,
bench::Options::FO_TEXT,
0));
std::stringstream name_str;
name_str << "benchmark_" << maxn;
std::string name = name_str.str();
BENCH_SAVE(name + ".out", bench::Options(bench::Options::VE_NORMAL,
bench::Options::FO_TEXT,
0));
BENCH_SAVE(name + ".xml", bench::Options());
BENCH_SAVE(name + ".dot", bench::Options(bench::Options::VE_MINIMAL,
bench::Options::FO_DOT));
}
BENCH_PRINT(bench::Options(bench::Options::VE_NORMAL,
bench::Options::FO_TEXT,
0));
BENCH_SAVE(name + ".out", bench::Options(bench::Options::VE_NORMAL,
bench::Options::FO_TEXT));
BENCH_SAVE(name + ".xml", bench::Options());
BENCH_SAVE(name + ".dot", bench::Options(bench::Options::VE_MINIMAL,
bench::Options::FO_DOT));
BENCH_SAVE(name + ".plot", bench::Options(bench::Options::VE_NORMAL,
bench::Options::FO_GNUPLOT,
bench::Options::CO_MEMPLOT));
}
cbs/demo/timer-bench/Makefile.am
View file @ 49b1d5b5
......@@ -18,3 +18,6 @@ noinst_PROGRAMS = demo-timer-bench
demo_timer_bench_SOURCES = main.cc
demo_timer_bench_LDADD = $(top_builddir)/lib/libbench.la
demo_timer_bench_CPPFLAGS = $(AM_CPPFLAGS)
bench: all
./demo-timer-bench
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment