The passive cooling technique is a strategy to improve building thermal performance which is cost-effective, eco-friendly and best suited for the local climate. The building material is one of the elements in passive cooling techniques. The research aims to investigate the influence of building materials on building thermal performance by field measurements on test houses. Hobo data loggers were used to collect indoor air temperature and relative humidity data. FLIR Infrared Camera was used to collect surface temperature data. The National Standard for Ventilation and Air Conditioning system (SNI 03-6572-2001) used as a reference for thermal performance value. The scope of the research is investigating the influence of the structure and wall materials on building thermal performance. The structure materials for the experiment were steel material and concrete material. The wall materials for this experiment were clay brick, light concrete brick (AAC), GRC-Rockwool panel and GRC-Styrofoam Panel. This research showed that steel structure influences the increasing indoor building air temperature. Clay brick material with higher thermal conductivity values has longer duration above the warm comfort zone rather than other wall materials in this experiment.