2
$\begingroup$

I'd like to be able to view an answer I gave to a question that it seems the person who asked the question has deleted. How do I get to this?

$\endgroup$
  • $\begingroup$ Do you remember the title of the question? Maybe I can look it up and write it here foryou :) $\endgroup$ – Dawny33 Jan 22 '16 at 7:09
  • $\begingroup$ Nope. Don't remember. However, I now I have the words Mathematica, Classify, and TopProbabilities (as one word) in the text of my answer. $\endgroup$ – Edmund Jan 22 '16 at 13:39
  • $\begingroup$ @Dawny33 I found the post link as I got a badge from it. 6766 However, it still shows as deleted so I can't see it even though I have the link. I think you should still be able to see your answer to a post in your activity even if the poster deletes the post. $\endgroup$ – Edmund Jan 24 '16 at 13:16
  • $\begingroup$ Added your answer as an answer here :) $\endgroup$ – Dawny33 Jan 24 '16 at 13:20
1
$\begingroup$

Your answer to the deleted question

Question: Machine learning algorithm to predict product service requests

Mathematica code below as I could not get it to work in R and OP asked to see what was done with Mathematica. More information on any of the functions can be found at the Wolfram Language & System Documentation Center. If you are wondering all functions/packages there are included standard with Mathematica so no need to purchase additional packages.

Import all rows and columns 1, 2, 7, and 9 from the first sheet of the spreadsheet.

data = Import[
   FileNameJoin[{NotebookDirectory[], "Sample_Set.xlsx"}], 
    {"XLSX", "Data", 1, All, {1, 2, 7, 9}}];

Clean items that are missing at least column entry. Also, drop header row.

cleanedData = Select[! Or @@ ((# == "") & /@ # ) &]@Rest@data;

(# == "") & is a nested pure function that is Maped (/@) over each entry of each row to create a list of equality test to "". Or is Applyed (@@) to the list and negated (!) to Select only those rows that have an entry in all columns. Rest drops the first row and keeps the rest.

Format into example->class for Classify function. The customer request comment is split into words and restricted to words that can be found in the dictionary. The {action, part} pairs are joined into one field action+":"+part.

classifyFormatted = (
     {
       Select[Length[DictionaryLookup[#, 1]] > 0 &]@TextWords@ToLowerCase[#[[1]]], #[[2]]
     } -> #[[3]] <> ":" <> #[[4]]) & /@ cleanedData;

Map over each row of cleanedData to create a rule per row of
{list of dictionary words from comment, issue} -> action+":"+part

Drop items for which no dictionary words were found in the customer request.

cleanedClassifyFormatted = Select[classifyFormatted, Length[#[[1, 1]]] != 0 &];

For each row check the length of the first sub-item of the first item (the list of dictionary words) is not zero.

Remove duplicate entries.

cleanedClassifyFormatted = DeleteDuplicates[cleanedClassifyFormatted];

Now we are left with 427 items from the original 499.

Dimensions@cleanedClassifyFormatted
(* 427 *)

Use Classify to create a ClassifierFunction. I let the selection algorithm choose the best method instead of explicitly selecting one. I also leave the last 27 items to test the classifier by passing in from the 1st to the 28th-last items only.

c1 = Classify[cleanedClassifyFormatted[[1 ;; -28]]];
ClassifierInformation[c1]

enter image description here

The output tells us that there are 136 classes from 400 observations based on 2 features.

Test classification on last 27 entries.

Transpose@{c1 /@ cleanedClassifyFormatted[[-27 ;;]][[All, 1]], 
   cleanedClassifyFormatted[[-27 ;;]][[All, 2]]} // TableForm

enter image description here

As you can see it does not predict very well. Looking at the Tally of the top ten classes we see that there may not be enough support for all classes.

(SortBy[-#[[2]] &]@Tally[cleanedClassifyFormatted[[1 ;; -28, 2]]])[[
  1 ;; 10]] // TableForm

enter image description here

Examining how may items have a particular tally we see that the is not enough support across all classes for a reasonable classification.

TableForm[
 Sort@Tally[Tally[cleanedClassifyFormatted[[1 ;; -28, 2]]][[All, 2]]],
 TableHeadings -> {None, {"Tally", "Num Items"}}]

enter image description here

Most of the classes have one observation so and only 1 has 41 observations. The sample does not have enough observations for the pairs.


Update

Added just classifying the part used to give get more support for the classes.

Reused cleanedClassifyFormatted and just StringDroped the part before the ":" from the rule of each entry.

cCF2 = #[[1]] -> StringDrop[#[[2]], StringPosition[#[[2]], ":", 1][[1, 1]]] & /@ cleanedClassifyFormatted;

Then we can execute the same commands as above to classify with this set.

Top ten Tally

(SortBy[-#[[2]] &]@Tally[cCF2[[1 ;; -28, 2]]])[[1 ;; 10]] // TableForm

enter image description here

Number of items per Tally.

TableForm[Sort@Tally[Tally[cCF2[[1 ;; -28, 2]]][[All, 2]]],
 TableHeadings -> {None, {"Tally", "Num Items"}}]

enter image description here

ClassifierFunction construction.

c2 = Classify[cCF2[[1 ;; -28]]];
ClassifierInformation[c2]

enter image description here

Here I'll switch it up by showing the top 3 probabilities from the ClassifierFunction for 5 of the test data sets and compare to the actual class.

Grid[Transpose@{Column@c2[#, {"TopProbabilities", 3}] & /@ 
    cCF2[[-27 ;;]][[1 ;; 5, 1]], cCF2[[-27 ;;]][[1 ;; 5, 2]]}, 
 Background -> {None, {{None, LightBlue}}} ]

enter image description here

The free text with sample size is the biggest limiter for this. I've attempted to create some structure with it by keeping only words that are found in the dictionary but there are still a lot of variations there. If you have them, I would suggest adding more structured fields like type of appliance, age of appliance, brand, and so on. These better define the classes.

You should also read up on the Method options of Classify and select the ones best suited for your data.

$\endgroup$
  • 1
    $\begingroup$ Many thanks, Dawny. :-) $\endgroup$ – Edmund Jan 24 '16 at 13:24
0
$\begingroup$

Unfortunately, I don't think that can be done. However, have a look at this meta discussion.

But, the moderators can see deleted posts.

$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .