I have loads of questions about the practicalities of storing and backing up large datasets, about things like how to select appropriate hardware and filesystems for my particular use case, how best to make use of cloud storage services etc. (i.e. questions directly related to storage rather than analysis).

Are these sorts of things on topic for Data Science SE? If not, where might they belong?

  • 1
    $\begingroup$ How about serverfault.com ? $\endgroup$ – Sean Owen Dec 13 '14 at 14:06
  • $\begingroup$ @SeanOwen Yeah, I've tried asking similar questions at ServerFault in the past. Whilst ServerFault is a good place to ask about generic storage/backup for commercial data, a suitable solution for us has to be tailored to the way in which we store and process our datasets. For example, we're using HDF5 rather than, say, SQL databases, which draws totally blank looks from ServerFault crowd. I'd really like to get the opinions of people who are used to dealing with the specific problems associated with storing and backing up large scientific datasets. $\endgroup$ – ali_m Dec 13 '14 at 14:25

I'd imagine that you wouldn't get much help about hardware of any kind here, or system-level tuning, or cloud services. If it's really about storage, not sure this is the place. If it's about how architect data storage to work efficiently with machine learning / predictive analytics, that feels more on topic for the audience knocking around here.


You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .