środa, 6 lipca 2011

enum <-> string

Nice preprocessor usage (found here: http://lists.gnucash.org/pipermail/gnucash-devel/2005-March/012849.html)


#include
#define ENUM_BODY(name, value) \
name = value,
#define AS_STRING_CASE(name, value) \
case name: return #name;
#define FROM_STRING_CASE(name, value) \
if (strcmp(str, #name) == 0) { \
return name; \
}
#define DEFINE_ENUM(name, list) \
typedef enum { \
list(ENUM_BODY) \
}name; \
const char* asString(name n) { \
switch (n) { \
list(AS_STRING_CASE) \
default: return ""; \
} \
} \
name fromString(const char* str) { \
list(FROM_STRING_CASE) \
return 0; /* assert? throw? */ \
}
#define ENUM_LIST(_) \
_(RED, 0) \
_(GREEN, 1) \
_(BLUE, 87)

DEFINE_ENUM(Color, ENUM_LIST)

int main() {
Color c = GREEN;
printf("%d\n",c);
printf("%s\n", asString(c));
printf("%d\n", fromString("BLUE"));
}


Another enum -> string can be found here

niedziela, 22 maja 2011

Memory allocation checking

Here is list how to check for memory allocation problems (leaks, writing past buffer, etc.)

glibc based:
- mcheck(), mprobe() functions and MALLOC_CHECK_ env. variable (see 'man malloc' and http://www.gnu.org/s/libc/manual/html_node/Heap-Consistency-Checking.html)

- mtrace() function and 'mtrace' command http://www.gnu.org/s/libc/manual/html_node/Allocation-Debugging.html

- glibc-utils (mtrace, xtrace, memusage, ...)


gcc based:
- Mudflap Pointer Debugging http://gcc.gnu.org/wiki/Mudflap_Pointer_Debugging

- GCC extensions: Bounds checking patches, ProPolice Stack-Smashing Protector
One thing is not clear here - there is option -fbounds-check which seems to compile in gcc (does it work?) and 'bounds checking patches' which gives -fbounds-checking option.


One and only:
- valgrind


Other free solutions:
- Electric fence http://directory.fsf.org/project/ElectricFence/

- DUMA library http://sourceforge.net/projects/duma/

- Memwatch http://www.linkdata.se/sourcecode/memwatch/

- halloc - not really a checker but useful http://directory.fsf.org/project/halloc/


Proprietary:
- Insure++

- Purify (IBM-Rational)


There is basic question - maybe design approach should be different e.g.:
- allocate memory on startup (pool)
- use sentry to check if tasks release objects allocated on pool

niedziela, 15 maja 2011

Here are potentially useful libraries used in Samba project.

TDB (Trivial DataBase)
"In concept, it is very much like GDBM, and BSD's DB except that it allows multiple simultaneous writers and uses locking internally to keep writers from trampling on each other. tdb is also extremely small."
It has added transactions support.
http://tdb.samba.org/index.html

TAlloc
"Hierarchical, reference counted memory pool system with destructors. It is the core memory allocator used in Samba (LGPL)."
http://talloc.samba.org/talloc/doc/html/index.html
man 3 talloc

TEvent
"Tevent is an event system based on the talloc memory management library."
http://tevent.samba.org/index.html

All Samba related sub-projects one can find at http://samba.com under "Related Sites".

Additionally here is page with junk code form Samba author:
http://www.samba.org/junkcode/
More description of junk code in conference materials.
Especially worth mentioning is 'segv_handler'.

Parallel mode extensions in gcc standard c++ library

Approach based on OpenMP. Here is info regarding the extension:
http://gcc.gnu.org/onlinedocs/libstdc++/manual/parallel_mode.html

wtorek, 1 marca 2011

Regexes for xml parsing

Limitations of presented resolution:
- regexes - shall be used only for "uncertain" data (e.g. xmls are not well formed)
for "real" xml real parser shall be used (e.g. expat)
- elements structure where child element has same name is not allowed e.g.
<a><a></a></a>
- empty-element tags are not recognized e.g. <a/>

XML declaration - search for encoding
"<\\?xml(\\s+(?:[^\\?<>]*?\\s+)*encoding\\s*=\\s*(['\"])((?:(?!\\2).)*)\\2[^\\?<>]*)\\?>"
Result groups:
1 - attributes
3 - encoding attribute value

Element with arbitrary name
"<([^\\s<>]+)(?:(\\s[^<>]*)?>(.*?)</\\1)?\\s*>"
Result groups:
1 - element name
2 - attributes
3 - element value

Element with specified name
"<(" + elem_name + ")(\\s[^<>]*)?>(.*?)</" + elem_name + "\\s*>"
Result groups:
1 - element name
2 - attributes
3 - element value

Element with specified name and required attribute
"<(" + elem_name + ")(\\s+(?:[^<>]*?\\s+)*" + attr_name + "\\s*=\\s*(['\"])((?:(?!\\3).)*)\\3[^<>]*)>(.*?)</" + elem_name + "\\s*>"
Result groups:
1 - element name
2 - attributes
4 - required attribute value
5 - element value

Element with specified name and optional attribute
"<(" + elem_name + ")(\\s*>|\\s+(?:[^<>]*?\\s+)*(?:" + attr_name + "\\s*=\\s*(['\"])((?:(?!\\3).)*)\\3)?[^<>]*)>(.*?)</" + elem_name + "\\s*>"
Result groups:
1 - element name
2 - attributes
4 - optional attribute value
5 - element value

Search for attribute within attribute result from element parsing
"\\s+" + attr_name + "\\s*=\\s*(['\"])(.*?)\\1"
Result group 2 - attribute value

Here is discussion on stackoverflow regarding the regexes for xml:
http://stackoverflow.com/questions/5204022/regex-for-xml-parsing

czwartek, 24 lutego 2011

CMMB and not well-formed xml

Chinese mobile TV standard CMMB contains data in xml format.
Unfortunately broadcasters send data in files that are not well-formed xmls.
It is common that ampersand sign '&' is not in entity form '&amp;'.
Who knows what else can we find there...

Now I know that there is lot more:
- time is crazy, especially time shift from UTC, sometimes it is +8h, sometimes -8h, sometimes 0, different across country with special "cases" in Hong-Kong and Macau,
- moreover time in DTMB seems to be delayed from CMMB (and correct time) for ~15min. in Shanghai,
- EPG are not updated properly, sometimes delayed,
- ...

niedziela, 9 sierpnia 2009

Priority inversion interview

Interviews are great possibility to evalute one's own memory and cold blood during conversation. It is also good for remembering of some basic terms and problems.
Recentely I had to describe priority inversion problem. Basic stuff :) thread with lower priority is executed in place of higher priority thread. But why? Wait, ..., well, ..., shit, I do not remember.
Why Wikipedia is not connected to my brain - 3 threads, 2 competing for mutex, third executing, and so on (Mars Pathfinder problem, priority inheritance, priority ceiling, disabling interrupts).
Ok, but if I want to simulate such problem in Windows environment?
After quick search I found Priority Inversion and Windows NT Scheduler. I realized that:
1. real-time priority class shall be set for process - to disable kernel altering threads priorites,
2. example shall run on one core - in simple case of 3 threads,
3. on one core machine system will hang (real-time priority), therefore example can be run only on multi-core machine (but threads will use only one of the cores).
Example code for priority inversion:


class PrioriyInversion
{
static private object o = new object();

static void tf(object p)
{
string n = (string)p;
Console.WriteLine(p+" critical section needed");
lock (o)
{
Console.WriteLine(p+" critical section entered");
Thread.Sleep(5000);
Console.WriteLine(p+" after sleep");
}
Console.WriteLine(p+" critical section left");
}

static void tf2(object p)
{
string n = (string)p;
Console.WriteLine(p + " start");
for (int i = 0; i < 1000000; ++i)
for (int j = 0; j < 1000000; ++j)
;
Console.WriteLine(p + " stop");
}

static void Main(string[] args)
{
Console.ReadLine();

Thread t1 = new Thread(tf);
t1.Priority = ThreadPriority.BelowNormal;
Thread t2 = new Thread(tf);
t2.Priority = ThreadPriority.AboveNormal;
Thread t3 = new Thread(tf2);
t3.Priority = ThreadPriority.Normal;

t1.Start("t1");
Thread.Sleep(10);
t2.Start("t2");
t3.Start("t3");
}
}


Program has Console.ReadLine() at the beginning to let user change affinity to one of the cores only and set priority class of the process to real-time. If these conditions are not achieved, priority inversion will not appear.
Additionally to change affinity and priority Windows Task Manager can be used. But if you want to see threads inside process, Process Explorer from Sysinternals (now on Microsoft page) can be used.