Data can be downloaded from here or here (packed 7.5M, unpacked 1.2G).
The archive contains 5 different datasets, corresponding to 5 question topics: color, horizontal location, vertical location, shape and size.
The img.amat files contain the 32x32 images: each line lists the gray level of the 1024 pixels in ASCII
The txt files contain the question and answer (for the respective image) in ASCII.
There are 10000 image-question pairs in the train files (the training set) and 5000 pairs each in the valid and test files (the validation and test sets).