Mechanical oscillators coupled to an electromagnetic cavity have emerged as a new frontier in quantum optics. This coupling presents an opportunity for manipulating the quantum state of light, connecting different quantum resources, and ultra-sensitive force detectors. A particular enabling platform utilizes high-stress silicon-nitride membrane resonators. Because of its large tensile stress, the membrane exhibits remarkable mechanical quality factors (Qs) that are even higher than that of single crystalline silicon. The membrane with such high Q combined with robust cryogenic Fabry-Perot cavity enables a variety of quantum optics experiments, such as squeezed light generation. Improving upon these encouraging results requires understanding and engineering the membrane resonators. The membrane dissipation can be classified as the internal loss and the external loss. We are able to manage the internal loss of a hybrid membrane with a curvature map, and control the external loss of a membrane with a phononic crystal (PnC) shield. In this dissertation, I will present squeezed light generation, our studies and engineering of high-stress membrane mechanics, and optomechanical Raman-ratio thermometry with a PnC-isolated membrane.