As suggested by Luke I set all parameters and settings back to their defaults and also increased the available swap space. Then ran the benchmark again but got similar results. TPC-H power test was about 20 seconds faster and during the throughput test (5 parallel streams) I got even more out of memory errors. These last ones are strange since there was still ample memory and swap space available when the errors occurred.
so for now and as far as I'm concerned: Greenplum case closed.
