14
$\begingroup$

There is a common question type that gets down voted or closed here. Rightly so in my opinion. However, I think the steady stream of similar questions needs somewhere we can point them.

Typical scenario is:

  • A beginner to machine learning has a data set, imagined or real, and a problem to solve.

  • As a beginner, they are confused about selecting the "best" modelling approach. There are loads of choices to make after all, and most involve learning about the model class and an API for it in some framework.

  • The question is presented better than "Here's some data, what do I do?" or "Give me the model?" questions in that the poster is aware of some options in front of them and has a real problem that they are trying to solve themselves. However, despite trying, they have still managed to post a question that is badly framed or too broad.

  • The question is at a fork in the road - either the poster could share more concrete details about what they are doing, or they could simply be encouraged to try and test their own ideas. Both could be acceptable outcomes, it is not 100% clear whether the problem needs more guidance than "What I'd do is try all 3 of your ideas and test them - so why don't you do the same?".

The reality is that even with a lot of experience, second-guessing the best model for a problem is not how things work. Instead an expert will typically explore the data and problem, select a metric to measure success, then try a range of possible models and feature engineering, testing each.

I think the asker's desire is to get some help through the model selection maze. However, they tend to forget how unique each data problem often is, and/or seem to think that more experienced ML experts will somehow know what to do, sometimes only with the problem domain, a hand-waving description of the data and a rough goal.

I'd like either this question, or even a canonical duplicate, to serve as a pointer to how the posters could improve their questions so that they can be answered and/or to be a frame challenge - i.e. the experts don't just read literature looking for "best solution to problem X", in fact they mostly just try and test stuff; the best thing is to learn the skills that enable the asker to do the same.


Some recent example questions:

$\endgroup$
1
$\begingroup$

When I first read your post, I agreed with it; but the more I think about it, the more I don't. In your post, you say:

Instead an expert will typically explore the data and problem, select a metric to measure success, then try a range of possible models and feature engineering, testing each.

This is certainly true. But can't more be said on each of these points? What does explore mean (and do different problems have data exploration)? What are typical metrics of success is each problem? What kind of features might be good to engineer for the problem at hand? With models, are there models to try as a baseline first?

I think there is a lot of process to unpack here. Certainly there are a lot of questions posed which have too few details; your complaint, however is about all such questions, not just vague the ones. If you remove all such questions, it looks to me like you're a step closer to making the questions here a subset of the questions of Cross-Validated. So in fact it seems that these questions are the kind that distinguish the board from Cross-Validated.

$\endgroup$
  • 2
    $\begingroup$ In fact I am not suggesting to remove the questions (I may post my own answer). The problem is that the askers are trying in the cases I link, but have not given enough information for a good answer. What advice would you offer to those OPs? Most importantly, is there in your opinion a standard piece of advice that covers a wide range of this type of question - perhaps similar to Stack Overflow's MVCE stackoverflow.com/help/mcve . . . something that helps the OP go from vague question about Data Science which model to use, to answerable question? $\endgroup$ – Neil Slater Apr 26 '17 at 15:00
  • 2
    $\begingroup$ @NeilSlater, I'd say there are some guidelines as to the minimum amount of detail to submit the question. Some ideas on what should be there: (1) goal of a predictive model; (2) description of the data, including (a) number of rows, (b) type of target, (c) type of possible inputs already in the model, and (3) description of what they have tried so far. $\endgroup$ – Paul Apr 26 '17 at 15:20
  • 2
    $\begingroup$ Great, do you think that could go into your answer? Because the goal of my question on Meta is to stimulate some ideas of what to do about these questions on the main site, and ultimately help people ask better questions. Data Science is a relatively quiet beta site, so IMO needs this kind of content, even when obvious. $\endgroup$ – Neil Slater Apr 26 '17 at 15:24
  • 2
    $\begingroup$ Absolutely, and at this stage of Data Science, there's almost nothing that's really "obvious." :) $\endgroup$ – Paul Apr 26 '17 at 15:40
1
$\begingroup$

What about having a page with examples of tasks and their categories to help people to categorize their tasks, and hopefully help them to model their problem?

Today I encountered a question, which of course is being flagged as too broad, but the problem is that the person didn't even know the term entity extraction.

Often I see this problem, where the person ask, "How do I find the trend?" or "How do I predict?" when often it is a "simple"classification, regression or clustering problem. However, the person cannot classify their problem (either because they don't know the terms or because they don't understand their own task).

And if they still cannot classify their problem, there is a list of information they need to provide to be pointed to the right direction. A list like Paul wrote in the comments of the other answer:

(1) goal of a predictive model;

(2) description of the data, including (a) number of rows, (b) type of target, (c) type of possible inputs already in the model, and

(3) description of what they have tried so far

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .