Here what I would personally do, might not work for you but should give some ideas at least.
1) Be clear what your game is about to avoid "unnecessary" feedback.
2) Release test builds often as possible.
3) Make sure players understand meaning of "preview build/dev build", "alpha" and "beta" (maybe even "RC, Release candidate").
Surveys are always nice idea after each testing, sometimes players have problems putting their feedback into words. Simple "rate feature" survey (example google drive forms) could bring more useful and interesting data compared to forum post "it was fun game".
When feedback is not so much about adding/removing features anymore, announce feature freeze and focus on polishing remaining "weak parts", but always remain honest with yourself. If you dont agree that feature X should be like Person A suggested, discuss with your players and provide them alternatives on how to improve feature X the way you feel would work better.
If you blindly follow every player suggestions you end up with "little bit of everything" and eventually game does not feel like its your own creation anymore, try to "stay true to your vision" while providing best possible experience for those who enjoy it.
You always have content/bug fix updates to add new features and tweak things even if game is not in beta anymore, when players are able to stress test your game they have better understanding what could be improved to make whole package work better.