As I have said elsewhere, learning the fretboard is difficult when it is divorced from playing songs and exercises actually using the fretboard positions you are trying the learn.
So playing the notes in question is very important. That’s why learning the fretboard goes hand in hand with knowing how to read traditional music, at least at a preliminary level. Relating the position on the staff to a position (actually, positions) on the fretboard gives you an important point of reference.
Here is an example of an exercise which has the objective of learning the positions of A and E in what I call the 1st segment of the fretboard – the part from the nut to fret 5.
Before you scoff at this as pointlessly academic, give it a try, and see if it helps give you a better understanding of the fretboard.
This exercise is from the MuzicTrain “Learning the Fretboard” series.