Wednesday, November 21, 2012

SAS: How to handle time and memory warning while using Fisher's Exact Test !

I have described below how to handle "WARNING: Computing exact p-values for this problem may require much time and memory. Press the system interrupt key to terminate exact computations."

A table contains a high proportion of small expected values (expected values less than 5).

In this case, we can use a Fisher’s exact test. Here are the commands we first try to use:

proc freq data = abc;
tables a*b / chisq expected;
exact fisher;
run;

The following message in the SAS log warns us that this may take a long time. We interrupted processing by clicking on the “break” key, which looks like a circle around an exclamation point (!).

150 proc freq data = abc;
151 tables a*b / chisq;
152 exact fisher;
153 run;

WARNING: Computing exact p-values for this problem may require much time and memory. Press the
system interrupt key to terminate exact computations.
NOTE: There were 405 observations read from the data set abc.
NOTE: PROCEDURE FREQ used (Total process time):
real time 31.02 seconds
cpu time 23.54 seconds

We now resubmit the commands, using instead the Monte Carlo option in SAS (mc). This will give us a quite good approximation to the Fisher’s exact test p-value, but based on 10,000 randomly chosen tables.

proc freq data = abc;
tables a*b / chisq expected;
exact fisher / mc;
run;

The Below PharmaSUG Paper also has a nice macro that can be used to handle the Fisher's warning :-

http://www.pharmasug.org/proceedings/2012/PO/PharmaSUG-2012-PO05.pdf

**** Knowledge Increases By Sharing. So, Pass it On ****

1 comment: