What I meant by the title is, what is the difference between my implementation, which I believe is close to the original rmsprop, and the keras’ implementation.

Kera’s fit method gets a loss < 0.01 in 10 epochs, while my naive algorithm slows down quite a lot after the 0.1 mark, and never reaches 0.01 even after 100 epochs.

### KERAS:

Epoch 1/10

1000/1000 [==============================] - 3s 3ms/step - loss: 0.7463 - accuracy: 0.7680

Epoch 2/10

1000/1000 [==============================] - 3s 3ms/step - loss: 0.3919 - accuracy: 0.8980

Epoch 3/10

1000/1000 [==============================] - 2s 2ms/step - loss: 0.2604 - accuracy: 0.9420

Epoch 4/10

1000/1000 [==============================] - 2s 2ms/step - loss: 0.2144 - accuracy: 0.9550

Epoch 5/10

1000/1000 [==============================] - 2s 2ms/step - loss: 0.1174 - accuracy: 0.9730

Epoch 6/10

1000/1000 [==============================] - 2s 2ms/step - loss: 0.0748 - accuracy: 0.9840

Epoch 7/10

1000/1000 [==============================] - 2s 2ms/step - loss: 0.0459 - accuracy: 0.9870

Epoch 8/10

1000/1000 [==============================] - 2s 2ms/step - loss: 0.0196 - accuracy: 0.9940

Epoch 9/10

1000/1000 [==============================] - 2s 2ms/step - loss: 0.0107 - accuracy: 0.9970

Epoch 10/10

1000/1000 [==============================] - 2s 2ms/step - loss: 0.0061 - accuracy: 0.9970

### NAIVE IMPLEMENTATION:

epoch: 0 loss: 2.351585176975065 acc: 0.073 // no training before this

epoch: 1 loss: 1.3441321235522397 acc: 0.738

epoch: 2 loss: 0.8169945169854006 acc: 0.822

epoch: 3 loss: 0.5958281075546342 acc: 0.849

epoch: 4 loss: 0.48944367167816155 acc: 0.868

epoch: 5 loss: 0.4215665684223718 acc: 0.879

epoch: 6 loss: 0.3784920125976777 acc: 0.891

epoch: 7 loss: 0.3491281265388871 acc: 0.899

epoch: 8 loss: 0.3267179105908342 acc: 0.902

epoch: 9 loss: 0.29812428350751236 acc: 0.917

epoch: 10 loss: 0.2816787871403444 acc: 0.918

epoch: 11 loss: 0.26484099127320365 acc: 0.922

epoch: 12 loss: 0.25452064445405986 acc: 0.928

epoch: 13 loss: 0.24510738403612584 acc: 0.925

epoch: 14 loss: 0.2313635832678401 acc: 0.934

epoch: 15 loss: 0.2263060342080479 acc: 0.934

epoch: 16 loss: 0.2141716029870379 acc: 0.937

epoch: 17 loss: 0.20585278438998866 acc: 0.941

epoch: 18 loss: 0.2008210906589328 acc: 0.939

epoch: 19 loss: 0.1914643542433619 acc: 0.943

epoch: 20 loss: 0.18526972395147176 acc: 0.95

epoch: 21 loss: 0.17977634057074943 acc: 0.953

epoch: 22 loss: 0.17659312338459698 acc: 0.95

epoch: 23 loss: 0.1730095272271334 acc: 0.951

epoch: 24 loss: 0.16751114601528463 acc: 0.956

epoch: 25 loss: 0.1622696005318275 acc: 0.956

epoch: 26 loss: 0.1584748890775764 acc: 0.96

epoch: 27 loss: 0.1541660845336603 acc: 0.963

epoch: 28 loss: 0.15366931850717738 acc: 0.962

epoch: 29 loss: 0.14791503167922196 acc: 0.964

epoch: 30 loss: 0.14370620733897888 acc: 0.965

epoch: 31 loss: 0.1394299197253062 acc: 0.967

epoch: 32 loss: 0.13728666423867625 acc: 0.968

epoch: 33 loss: 0.13302949320498997 acc: 0.968

epoch: 34 loss: 0.13020528536144707 acc: 0.968

epoch: 35 loss: 0.13069805470523502 acc: 0.969

epoch: 36 loss: 0.1276391479303531 acc: 0.97

epoch: 37 loss: 0.12226603405970293 acc: 0.969

epoch: 38 loss: 0.12101554054811792 acc: 0.969

epoch: 39 loss: 0.11890066609131254 acc: 0.969

epoch: 40 loss: 0.11783996723830573 acc: 0.971

epoch: 41 loss: 0.1132539505108236 acc: 0.97

epoch: 42 loss: 0.11192076822162904 acc: 0.973

epoch: 43 loss: 0.10894143290231988 acc: 0.972

epoch: 44 loss: 0.10717285655939912 acc: 0.974

epoch: 45 loss: 0.10487730744173353 acc: 0.974

epoch: 46 loss: 0.10197636382729229 acc: 0.973

epoch: 47 loss: 0.0991876673474291 acc: 0.973

epoch: 48 loss: 0.099348139794124 acc: 0.975

epoch: 49 loss: 0.09520582580655605 acc: 0.975

epoch: 50 loss: 0.0969406397611115 acc: 0.976

epoch: 51 loss: 0.09059255100317501 acc: 0.976

epoch: 52 loss: 0.09316977521888427 acc: 0.976

epoch: 53 loss: 0.08938247631626035 acc: 0.975

epoch: 54 loss: 0.08811868742037693 acc: 0.978

epoch: 55 loss: 0.08625686202783996 acc: 0.978

epoch: 56 loss: 0.08538771887459436 acc: 0.979

epoch: 57 loss: 0.08185635352133913 acc: 0.978

epoch: 58 loss: 0.08170590581949694 acc: 0.979

epoch: 59 loss: 0.07719183930779538 acc: 0.979

epoch: 60 loss: 0.07684899163736585 acc: 0.979

epoch: 61 loss: 0.07588368055325305 acc: 0.98

epoch: 62 loss: 0.07507468920577076 acc: 0.979

epoch: 63 loss: 0.07207410799842978 acc: 0.979

epoch: 64 loss: 0.07040289474188392 acc: 0.979

epoch: 65 loss: 0.0703488063887447 acc: 0.98

epoch: 66 loss: 0.06929370403706761 acc: 0.979

epoch: 67 loss: 0.0657441442659503 acc: 0.979

epoch: 68 loss: 0.06729136911819426 acc: 0.981

epoch: 69 loss: 0.06414198279278469 acc: 0.983

epoch: 70 loss: 0.0615023553909231 acc: 0.983

epoch: 71 loss: 0.06018738520679154 acc: 0.982

epoch: 72 loss: 0.05918258034472605 acc: 0.982

epoch: 73 loss: 0.056588497296133494 acc: 0.983

epoch: 74 loss: 0.059048515572232146 acc: 0.984

epoch: 75 loss: 0.054759201485826324 acc: 0.983

epoch: 76 loss: 0.052757782436277205 acc: 0.985

epoch: 77 loss: 0.05371287689539768 acc: 0.983

epoch: 78 loss: 0.05071757667213161 acc: 0.984

epoch: 79 loss: 0.04923249682405242 acc: 0.986

epoch: 80 loss: 0.0493799899097154 acc: 0.985

epoch: 81 loss: 0.04733707437497998 acc: 0.985

epoch: 82 loss: 0.04974538426387033 acc: 0.986

epoch: 83 loss: 0.04644481612691435 acc: 0.987

epoch: 84 loss: 0.04487185519164782 acc: 0.986

epoch: 85 loss: 0.045398671290498294 acc: 0.989

epoch: 86 loss: 0.04399450836766221 acc: 0.989

epoch: 87 loss: 0.0420689016845811 acc: 0.989

epoch: 88 loss: 0.04011364587751942 acc: 0.991

epoch: 89 loss: 0.039923482281579464 acc: 0.989

epoch: 90 loss: 0.03845436789415447 acc: 0.992

epoch: 91 loss: 0.03867456867975187 acc: 0.991

epoch: 92 loss: 0.03779903707806368 acc: 0.989

epoch: 93 loss: 0.0373895581236269 acc: 0.991

epoch: 94 loss: 0.03760058053188023 acc: 0.99

epoch: 95 loss: 0.03589038914134395 acc: 0.993

epoch: 96 loss: 0.03561026022566131 acc: 0.993

epoch: 97 loss: 0.03553280058120401 acc: 0.993

epoch: 98 loss: 0.03513232650415162 acc: 0.994

epoch: 99 loss: 0.03376538200643341 acc: 0.993

epoch: 100 loss: 0.03136016808749243 acc: 0.995

The implementation is in this repo: https://github.com/o-clipe/mykeraslike

Thank you in advance!