The first approach is the logical thing for a database design. Having a lot of data is not a consideration while designing a database. The essence of a database design is to model generic entities which can assume several representable values.
In your case, you are trying to store cities and districts and model their relationship. Your problem statement defines two generic entities (usually identified by nouns) - cities and districts. London, Liverpool, Manchester are all city names each of which is a specific data row of the cities table. Data is related to some other data, like in your case the data in cities has to be related to data in the district itself.
Your second approach is not relational modelling as it has been defined in the book. What you are trying to do is some kind of a direct lookup where the data is linked to an entity (I would term that as a lookup). The interesting thing to note here is that if you have table by the city names, you would have several tables with the same logical design and each containing one row. You might realize that this is pure redundancy. You would be putting related things in unrelated entities when they belong to the same class of data.
Pardon my explanation because I haven't any database specific terms here. I just wrote whatever logically came to my mind.