My submission: https://jovian.ml/glmiotto/03-cifar10-feedforward
I tried four models. The last one got 99.69% accuracy, however as I didn’t experiment a lot with varying epochs, this was likely a bit brute-force since I used a 6-layer structure (output sizes 1024, 512, 256, 128, 32, 10).
I was afraid the model would not focus on the important stuff with these node sizes but it worked out, though training was understandably a bit slow.
I used decreasing learning rates starting fairly high at 0.6 and going down to 0.005 until the final plateau. Always 5-10 epochs per alpha. Would love some input about how to design a good learning-rate / epoch number progression in a way that uses fewer layers.
Edit: well, duh. I did notice that the last training-validation set results were equal to the testing set results but didn’t really look into why - I guess the 99% accuracy was too juicy to question. Turns out the loaders all had the same data…
I’ll re-submit today with the update @aakashns, thank you for clarifying.