If you are an app developer using Flutter, then Serverpod is a great match. An app server is obviously a server specifically tailored to the needs of apps (e.g. as opposed to being made for the web).
You're misunderstanding the project, Serverpod basically creates the following artifacts with their own CLI:
1. Creates a Flutter folder for you, similar to what 'Flutter create' does. It's pre-configured to be able to connect to your local Serverpod server.
2. Creates a server folder where you'd write your API's in Dart. Since Flutter is written in Dart too, this opens up the possibility to share code such as domain models.
3. Creates a client folder with auto-generated code. No manual code changes should be done here, that's the magic trick of this project. This project contains the methods that calls your API's, no need to write redundant code anymore in your Flutter project for each API you just made in your server (backend) project.
It comes with more interesting bells and whistles such as using Docker (compose) to bootstrap every service and also leveraging AWS with Redis, Postgres etc which is setup with Terraform. It goes against the Flutter mass where most of the devs are working with Firebase.
So to answer your question, the Flutter app does not need to be served. It can be, as you said yourself, a local running app that (e.g. phone or desktop). However, the server folder is the one needing to be served for obvious reasons (authentication, safe CRUD operations etc).
You can also watch the Serverpod 1.0 release keynote on Youtube here: https://youtu.be/QN6juNWW3js