Harris' hawk optimization (HHO) is a recent addition to population-based metaheuristic paradigm, inspired from hunting behavior of Harris' hawks. It has demonstrated promising search behavior while employed on various optimization problems, however the diversity of search agents can be further enhanced. This paper represents a novel modified variant with a long-term memory concept, hence called long-term memory HHO (LMHHO), which provides information about multiple promising regions in problem landscape, for improvised search results. With this information, LMHHO maintains exploration up to a certain level even until search termination, thus produces better results than the original method. Moreover, the study proves that appropriate tools for in-depth performance analysis can help improve search efficiency of existing metaheuristic algorithms by making simple yet effective modification in search strategy. The diversity measurement and exploration-exploitation investigations prove that the proposed LMHHO maintains trade-off balance between exploration and exploitation. The proposed approach is investigated on high-dimensional numerical optimization problems, including classic benchmark and CEC'17 functions; also, on optimal power flow problem in power generation system. The experimental study suggests that LMHHO not only outperforms the original HHO but also various other established and recently introduced metaheuristic algorithms. Although, the research can be extended by implementing more efficient memory archive and retrieval approaches for enhanced results.