Dynamics of a family of rational maps concerning renormalization transformation