5
$\begingroup$

I'm posting this to discuss whether questions about performance optimizations are off-topic in data science, and of course to see better definitions for such questions. My main relation with data science -- which I'm not sure if it does actually exist right now -- is via performance improvement/distributed computing. Some of the questions I've posted, which I consider to be related to performance optimizations, are:

My point is: since these are not programming questions, and rather performance issues/techniques to help processing very large databases, they would (to me) be better fit to this site than StackOverflow, for example. Again, I'm not sure about what exactly would limit such questions, and neither if they are actually on topic here in DataScience.

Hope the answers shed light on a better definition for "performance optimization" questions, as well as decide whether they are on topic.

$\endgroup$
1
  • 2
    $\begingroup$ We really need a better term than "hacking-skills" for this discussion to remain visible... Maybe you could call it "questions about performance optimization" $\endgroup$
    – Air
    Commented Jun 18, 2014 at 18:56

1 Answer 1

Reset to default
4
$\begingroup$

Performance is an important aspect of Data Science. Average programmers don't have to deal with performance issues that are due to large amounts of data. For example if somebody asks in SO what's an efficient way to store multi-million records and query for the existence of a record, I doubt that the typical answer will involve a Bloom Filter. If the performance issue is not caused by large amounts of data probably it is off-topic.

SO is so big that of course there are performance gurus that will be able to answer any kind of question. However if these same questions are of interest of a broader data-science audience I believe we should endorse them not exclude them.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .