Skeleton data represent human body joints. Human action recognition with Skeleton data has been widely conducted. Yet, the research requires significant time and efforts, given that those data need to be manually labeled. Also, the previous work has been implementing supervised learning with hand annotated labels. Supervised learning has enabled researchers to achieve remarkable improvements, but with new inputs it is less powerful than the given. Therefore, by adding contrastive learning network to the previously announced Semantic Guided Neural Network (SGN), we successfully implemented data ugmentation on skeleton data without losing any advantages that SGN has.