Oxide fiber-reinforced oxide matrix composites (Ox/Ox-CMCs) were fabricated by vacuum-assisted impregnation and pressureless sintering using a high-yield alumina precursor. The Ox/Ox-CMCs were sintered at 1100 and 1200 °C for 2 h. SEM and X-ray CT revealed that in both cases the matrix formed successfully within and between the woven fabrics. Vickers hardness of the sintered matrix was independent of the location within each composite. In a three-point bending test, a bending fracture at the tensile side occurred prior to shear fracture, suggesting that the matrix achieved good adhesion between the woven fabrics. The Ox/Ox-CMC sintered at 1100 °C showed higher tensile strength than that sintered at 1200 °C. This strength difference was attributed to their microstructures; the specimen sintered at 1100 °C showed crack deflection at the fiber/matrix interface, while that sintered at 1200 °C had a strongly adherent interface. Both composites showed notch-insensitive behavior in double-edge notch tensile tests.