Most people model their house in 3D instead of using a 2D image. The PolyLine if I remember right is showing you positions that are relative to the first point in the PolyLine not an absolute world space position. I had thought about changing it or allowing a toggle between relative and absolute but never got motivated to do it.
Modeling the house in 3D makes the most sense but I do not see any drawing tools to do that. I can do a house model in solidworks pretty quickly (including the uneven yard) but I do not see any model import functions. The scale seems to be in pixels as well so I am not sure how that translates into real world dimensions.
Points relative to the first point makes sense as well. I did a little test and I think that is correct.
This probably belongs in the feature request but a front/side/top/iso view buttons would make seeing it a bit simpler.