| --- Log opened Thu Mar 15 00:00:42 2018 | ||
| Farouk | Hi Everyone. I have a question about the Neural network API. From what I understand, CNeuralNetwork::train() trains the network to try to match the required output using gradient descent. However, is there a way to train the neural network using an **error** function? For example, define log loss using the outputs of the neural network and minimize log loss using an optimizer, or is that still not available? | 06:07 |
|---|---|---|
| -!- HeikoS [~heiko@bmi-airport-1.inf.ethz.ch] has joined #shogun | 10:02 | |
| -!- mode/#shogun [+o HeikoS] by ChanServ | 10:02 | |
| --- Log closed Thu Mar 15 12:23:06 2018 | ||
| --- Log opened Thu Mar 15 12:30:21 2018 | ||
| -!- shogun-toolbox [~shogun@7nn.de] has joined #shogun | 12:30 | |
| -!- Irssi: #shogun: Total of 16 nicks [4 ops, 0 halfops, 0 voices, 12 normal] | 12:30 | |
| -!- Irssi: Join to #shogun was synced in 8 secs | 12:30 | |
| Farouk | Hi Everyone. I have a question about the Neural network API. From what I understand, CNeuralNetwork::train() trains the network to try to match the required output using gradient descent. However, is there a way to train the neural network using an **error** function? For example, define log loss using the outputs of the neural network and minimize log loss using an optimizer, or is that still not available? | 13:42 |
| @HeikoS | Farouk: hi there, I am not sure, did you check the notebooks? | 17:49 |
| @HeikoS | otherwise only reading in the code will reveal that | 17:49 |
| @HeikoS | lisitsyn: you kow that? | 17:49 |
| @HeikoS | but if it is not possible to do that, I think it would be a nice extension patch :) | 17:57 |
| Farouk | Hi Heikos, sorry for the late reply. I did spend most of my day yesterday reading the Code base. From what I saw, the Neural Network was performing SGD (Or L-BFGS) by minimizing the squared error loss between the target and and the prediction. This is fine for training, but I feel its a bit restrictive if you just want to minimize any other cost function. I also saw Adam and RmsProp were implemented so it'd be a shame if we didn't use | 18:07 |
| Farouk | actually, the compute_gradient, and compute_errors for all output layers were computed with just the square error loss in mind (So for example, all the back prop equations were hard coded with only squared error in mind). I think if we can generalize this to take into accound any error loss that would be a nice project | 18:09 |
| Farouk | Like use the CLossFunction class | 18:09 |
| -!- HeikoS [~heiko@bmi-airport-1.inf.ethz.ch] has quit [Ping timeout: 264 seconds] | 18:21 | |
| -!- Farouk [81617d05@gateway/web/freenode/ip.129.97.125.5] has quit [Ping timeout: 260 seconds] | 22:00 | |
| -!- Farouk [81617d04@gateway/web/freenode/ip.129.97.125.4] has joined #shogun | 23:31 | |
| --- Log closed Fri Mar 16 00:00:43 2018 | ||
Generated by irclog2html.py 2.10.0 by Marius Gedminas - find it at mg.pov.lt!