Profiling

Basic profiling tool - gprof

Required '-pg' added for compilation and linking.

Profile results are stored in file 'gmon.out'. Default place for the file is working directory of the program being profiled. Important - process has to be closed with normal termination - exit() or end of main() function - profile file is saved on process termination.

Potentially interesting - environmental variables for gprof (not sure if working):
GMON_OUT_PREFIX, PROFDIR, PROFFLAGS.

sources:
http://sourceware.org/binutils/docs/gprof/
http://www.cs.utah.edu/dept/old/texinfo/as/gprof.html
http://sourceware.org/gdb/onlinedocs/gdb/


Other:
Tool from Intel - Pin
GNU cflow analyzes a collection of C source files and prints a graph, charting control flow within the program.
ltrace, ptrace, etrace, ftrace
SystemTap - http://sourceware.org/systemtap/examples/general/para-callgraph.stp - Observe, systemtap and oprofile updates
-finstrument-functions gcc
http://blog.superadditive.com/2007/12/01/call-graphs-using-the-gnu-project-debugger/

enum <-> string

Nice preprocessor usage (found here: http://lists.gnucash.org/pipermail/gnucash-devel/2005-March/012849.html)


#include
#define ENUM_BODY(name, value) \
name = value,
#define AS_STRING_CASE(name, value) \
case name: return #name;
#define FROM_STRING_CASE(name, value) \
if (strcmp(str, #name) == 0) { \
return name; \
}
#define DEFINE_ENUM(name, list) \
typedef enum { \
list(ENUM_BODY) \
}name; \
const char* asString(name n) { \
switch (n) { \
list(AS_STRING_CASE) \
default: return ""; \
} \
} \
name fromString(const char* str) { \
list(FROM_STRING_CASE) \
return 0; /* assert? throw? */ \
}
#define ENUM_LIST(_) \
_(RED, 0) \
_(GREEN, 1) \
_(BLUE, 87)

DEFINE_ENUM(Color, ENUM_LIST)

int main() {
Color c = GREEN;
printf("%d\n",c);
printf("%s\n", asString(c));
printf("%d\n", fromString("BLUE"));
}


Another enum -> string can be found here

Memory allocation checking

Here is list how to check for memory allocation problems (leaks, writing past buffer, etc.)

glibc based:
- mcheck(), mprobe() functions and MALLOC_CHECK_ env. variable (see 'man malloc' and http://www.gnu.org/s/libc/manual/html_node/Heap-Consistency-Checking.html)

- mtrace() function and 'mtrace' command http://www.gnu.org/s/libc/manual/html_node/Allocation-Debugging.html

- glibc-utils (mtrace, xtrace, memusage, ...)


gcc based:
- Mudflap Pointer Debugging http://gcc.gnu.org/wiki/Mudflap_Pointer_Debugging

- GCC extensions: Bounds checking patches, ProPolice Stack-Smashing Protector
One thing is not clear here - there is option -fbounds-check which seems to compile in gcc (does it work?) and 'bounds checking patches' which gives -fbounds-checking option.


One and only:
- valgrind


Other free solutions:
- Electric fence http://directory.fsf.org/project/ElectricFence/

- DUMA library http://sourceforge.net/projects/duma/

- Memwatch http://www.linkdata.se/sourcecode/memwatch/

- halloc - not really a checker but useful http://directory.fsf.org/project/halloc/


Proprietary:
- Insure++

- Purify (IBM-Rational)


There is basic question - maybe design approach should be different e.g.:
- allocate memory on startup (pool)
- use sentry to check if tasks release objects allocated on pool

Here are potentially useful libraries used in Samba project.

TDB (Trivial DataBase)
"In concept, it is very much like GDBM, and BSD's DB except that it allows multiple simultaneous writers and uses locking internally to keep writers from trampling on each other. tdb is also extremely small."
It has added transactions support.
http://tdb.samba.org/index.html

TAlloc
"Hierarchical, reference counted memory pool system with destructors. It is the core memory allocator used in Samba (LGPL)."
http://talloc.samba.org/talloc/doc/html/index.html
man 3 talloc

TEvent
"Tevent is an event system based on the talloc memory management library."
http://tevent.samba.org/index.html

All Samba related sub-projects one can find at http://samba.com under "Related Sites".

Additionally here is page with junk code form Samba author:
http://www.samba.org/junkcode/
More description of junk code in conference materials.
Especially worth mentioning is 'segv_handler'.

Parallel mode extensions in gcc standard c++ library

Approach based on OpenMP. Here is info regarding the extension:
http://gcc.gnu.org/onlinedocs/libstdc++/manual/parallel_mode.html

Regexes for xml parsing

Limitations of presented resolution:
- regexes - shall be used only for "uncertain" data (e.g. xmls are not well formed)
for "real" xml real parser shall be used (e.g. expat)
- elements structure where child element has same name is not allowed e.g.
<a><a></a></a>
- empty-element tags are not recognized e.g. <a/>

XML declaration - search for encoding
"<\\?xml(\\s+(?:[^\\?<>]*?\\s+)*encoding\\s*=\\s*(['\"])((?:(?!\\2).)*)\\2[^\\?<>]*)\\?>"
Result groups:
1 - attributes
3 - encoding attribute value

Element with arbitrary name
"<([^\\s<>]+)(?:(\\s[^<>]*)?>(.*?)</\\1)?\\s*>"
Result groups:
1 - element name
2 - attributes
3 - element value

Element with specified name
"<(" + elem_name + ")(\\s[^<>]*)?>(.*?)</" + elem_name + "\\s*>"
Result groups:
1 - element name
2 - attributes
3 - element value

Element with specified name and required attribute
"<(" + elem_name + ")(\\s+(?:[^<>]*?\\s+)*" + attr_name + "\\s*=\\s*(['\"])((?:(?!\\3).)*)\\3[^<>]*)>(.*?)</" + elem_name + "\\s*>"
Result groups:
1 - element name
2 - attributes
4 - required attribute value
5 - element value

Element with specified name and optional attribute
"<(" + elem_name + ")(\\s*>|\\s+(?:[^<>]*?\\s+)*(?:" + attr_name + "\\s*=\\s*(['\"])((?:(?!\\3).)*)\\3)?[^<>]*)>(.*?)</" + elem_name + "\\s*>"
Result groups:
1 - element name
2 - attributes
4 - optional attribute value
5 - element value

Search for attribute within attribute result from element parsing
"\\s+" + attr_name + "\\s*=\\s*(['\"])(.*?)\\1"
Result group 2 - attribute value

Here is discussion on stackoverflow regarding the regexes for xml:
http://stackoverflow.com/questions/5204022/regex-for-xml-parsing

CMMB and not well-formed xml

Chinese mobile TV standard CMMB contains data in xml format.
Unfortunately broadcasters send data in files that are not well-formed xmls.
It is common that ampersand sign '&' is not in entity form '&amp;'.
Who knows what else can we find there...

Now I know that there is lot more:
- time is crazy, especially time shift from UTC, sometimes it is +8h, sometimes -8h, sometimes 0, different across country with special "cases" in Hong-Kong and Macau,
- moreover time in DTMB seems to be delayed from CMMB (and correct time) for ~15min. in Shanghai,
- EPG are not updated properly, sometimes delayed,
- ...