Limit RAM usage while creating LMDB - caffe

Question

I am trying to create a LMDB with 6 concatenated images as labels. My python script looks like this:
in_db = lmdb.open('image-lmdb', map_size=int(1e12), writemap=True)
with in_db.begin(write=True) as in_txn:
for in_idx, in_ in enumerate(inputs):
im = np.array(Image.open('train_label_1/' + in_))
im = im[:,:,1]
c.append(im)
im = np.array(Image.open('train_label_2/' + in_))
im = im[:,:,1]
c.append(im)
im = np.array(Image.open('train_label_3/' + in_))
im = im[:,:,1]
c.append(im)
im = np.array(Image.open('train_label_4/' + in_))
im = im[:,:,1]
c.append(im)
im = np.array(Image.open('train_label_5/' + in_))
im = im[:,:,1]
c.append(im)
im = np.array(Image.open('train_label_6/' + in_))
im = im[:,:,1]
c.append(im)
d = np.array(c)
im_dat = caffe.io.array_to_datum(d)
in_txn.put('{:0>10d}'.format(in_idx), im_dat.SerializeToString())
in_db.close()
I have two problems:
How big should the map_size be?
I have round about 140,000 labels. Each image inside the label has a size of 45 x 45. Because I am using only one dimension of each image I guess that every pixel is one byte. So my guess would be that the map_size should be 45*45*6*140,000 Bytes. But if I set the map_size to this value, I get a MapFullError: mdb_put: MDB_MAP_FULL: Environment mapsize limit reached after 528 labels.
If I set the map_size to 1e12 I don't get an MapFullError that fast. But instead the RAM usage goes to nearly 100%. Is there a way to limit RAM usage when creating a lmdb?
EDIT
To reduce the RAM usage I tried to iterate over the lmdb as described here:
for idx in range(int(math.ceil(len(inputs)/1000.0))):
in_db = lmdb.open('image-lmdb', map_size=int(1e12))#, writemap=True
with in_db.begin(write=True) as in_txn:
for in_idx, in_ in enumerate(inputs[(1000*idx):(1000*(idx+1))]):
im = np.array(Image.open('train_label_1/' + in_))
im = im[:,:,1]
c.append(im)
im = np.array(Image.open('train_label_2/' + in_))
im = im[:,:,1]
c.append(im)
.
.
.
d = np.array(c)
im_dat = caffe.io.array_to_datum(d)
in_txn.put('{:0>10d}'.format(in_idx + idx * 1000), im_dat.SerializeToString())
in_db.close()
But still the RAM usage gets to 99% and the writing of the lmdb slows very much down.

Answer 1

I found my mistake. I forgot to reset the array c after each iteration.