|
• large dataset and matlab
Hi,
Im new to matlab, I have a large dataset (about 2000000 records) and I
decided to use neural networks for forecasting purposes.
reading data is not a problem. I use database toolbox and fetch data.
the problem is data is too much and I don't have time (and memory and computation resources) to watch matlab train my net for several years!
so I want to train it incrementaly (I hope incremental training is what I need). so I decided to test something in matlab first. I used
this code from matlab help:
load phoneme
p = con2seq(y); t = con2seq(t);
lrn_net = newlrn([-1 1],[8 1],{'tansig','purelin'});
lrn_net.trainFcn = 'trainbr';
lrn_net.trainParam.show = 5;
lrn_net.trainParam.epochs = 50;
[lrn_net] = train(lrn_net,p,t);
and I simply changed the last line to:
[lrn_net] = train(lrn_net,p(1:200),t(1:200));
and after the training was done:
lrn_net.adaptFcn = 'trainbr'
[lrn_net] = adapt(lrn_net,p(201:400),t(201:400));
y = sim(lrn_net,p);
plot(cell2mat(y));
well ,thats all, but I don't get the expected results, if I use adapt
for all the data it works fine , if I use train for all data , it works fine too. but with this kind of combination I get nowhere.
Did I get it wrong ? how can I feed my massive data to neural network?
|