PM Modi to inaugurate Commonwealth speakers' conference; Pakistan, Bangladesh to stay away