群体智能

2025年3月17日 | 阅读 14 分钟

群体智能是一个引人入胜的概念，它受自然界中去中心化、自组织系统的集体行为启发。它受到鱼、蚂蚁、蜜蜂和鸟类等许多动物中观察到的合作行为的影响，其中单个智能体与周围环境和彼此进行局部交互，以产生复杂协调的结果。研究人员一直被这种迷人的现象所吸引，这激发了在计算机科学、机器人学和优化等一系列领域中创建创新方法和算法来解决问题。

群体智能原理

去中心化控制：群体智能系统中，单个智能体的行动不受全局领导者或中心化控制的支配。相反，每个智能体都遵循基本规则，并利用局部数据与其邻居进行通信。
自组织：群体系统展现出自组织能力，其中复杂的全局模式通过简单的个体之间的局部互动和反馈过程而产生。在动态环境中，这种去中心化策略使得弹性适应行为成为可能。
涌现特性：群体智能系统经常展现出涌现特性，其中群体的整体行为超出了任何单个智能体所能做到的。高效的决策制定、优化和问题解决可以源于这些涌现特性。

代码

现在，通过实施群体智能，我们将尝试解决经典的哈利特问题。在资源管理游戏哈利特中，你建造并指挥一支小型舰队。你的算法跟踪它们的运动，以收集光能或哈利特。

创建哈利特环境

 
from kaggle_environments import evaluate, make

env = make("halite", debug=True)
env.render()

现在我们将遵循一种复杂的方法来为哈利特竞赛开发一个智能体，利用战略决策和基于记忆的学习，以在游戏环境中实现最佳性能。

 
#%%writefile submission.py
# for Debug and Evolutionary Algorithm previous line (%%writefile submission.py) should be commented out,
# uncomment to write submission.py

#FUNCTIONS###################################################
def get_map_and_average_halite(obs):
    """
        get an average amount of halite per halite source
        and map as a two-dimensional array of objects and set amounts of halite in each cell
    """
    game_map = []
    halite_sources_amount = 0
    halite_total_amount = 0
    for x in range(conf.size):
        game_map.append([])
        for y in range(conf.size):
            game_map[x].append({
                # value will be the ID of the owner
                "shipyard": None,
                # value will be the ID of the owner
                "ship": None,
                # value will be the amount of halite
                "ship_cargo": None,
                # amount of halite
                "halite": obs.halite[conf.size * y + x]
            })
            if game_map[x][y]["halite"] > 0:
                halite_total_amount += game_map[x][y]["halite"]
                halite_sources_amount += 1
    average_halite = halite_total_amount / halite_sources_amount
    return game_map, average_halite

def get_swarm_units_coords_and_update_map(s_env):
    """ get lists of coords of Swarm's units and update locations of ships and shipyards on the map """
    # arrays of (x, y) coords
    swarm_shipyards_coords = []
    swarm_ships_coords = []
    # Place on the map the locations of units of every player
    for player in range(len(s_env["obs"].players)):
        # Place on the map the locations of every shipyard of the player
        shipyards = list(s_env["obs"].players[player][1].values())
        for shipyard in shipyards:
            x = shipyard % conf.size
            y = shipyard // conf.size
            # place shipyard on the map
            s_env["map"][x][y]["shipyard"] = player
            if player == s_env["obs"].player:
                swarm_shipyards_coords.append((x, y))
        # Place on the map the locations of every ship of the player
        ships = list(s_env["obs"].players[player][2].values())
        for ship in ships:
            x = ship[0] % conf.size
            y = ship[0] // conf.size
            # place ship on the map
            s_env["map"][x][y]["ship"] = player
            s_env["map"][x][y]["ship_cargo"] = ship[1]
            if player == s_env["obs"].player:
                swarm_ships_coords.append((x, y))
    return swarm_shipyards_coords, swarm_ships_coords

def get_c(c):
    """ get coordinate, considering donut type of the map """
    return c % conf.size

def clear(x, y, player, game_map):
    """ check if the cell is safe to move in """
    # If there is no shipyard, or there is a player's shipyard
    # and there is no ship
    if ((game_map[x][y]["shipyard"] == player or game_map[x][y]["shipyard"] == None) and
            game_map[x][y]["ship"] == None):
        return True
    return False

def move_ship(actions, s_env, ship):
    """ move the ship in that direction """
    # If the ship has to move out from its current position
    if ships_data[ship["key"]]["direction_to_move"] != "HERE":
        actions[ship["key"]] = ships_data[ship["key"]]["direction_to_move"]
        for d in directions_list:
            if d["direction"] == ships_data[ship["key"]]["direction_to_move"]:
                x = d["x"](ship["x"])
                y = d["y"](ship["y"])
        ships_data[ship["key"]]["direction_to_move"] = "HERE"
        # apply changes to game_map, to avoid collisions of player's ships next turn
        s_env["map"][ship["x"]][ship["y"]]["ship"] = None
        s_env["map"][x][y]["ship"] = s_env["obs"].player
    return actions, s_env

def go_for_halite(s_env, ship):
    """ ship will go to a safe cell with enough halite if it is found """
    # If the current cell has enough halite
    if (s_env["map"][ship["x"]][ship["y"]]["halite"] > s_env["low_amount_of_halite"] and
            not hostile_ship_near(ship["x"], ship["y"], s_env["obs"].player, s_env["map"], ship["cargo"])):
        most_halite = s_env["map"][ship["x"]][ship["y"]]["halite"]
    else:
        # biggest amount of halite among scanned cells
        most_halite = s_env["low_amount_of_halite"]
    direction = None
    for d in range(len(directions_list)):
        x = directions_list[d]["x"](ship["x"])
        y = directions_list[d]["y"](ship["y"])
        # If the cell is safe to move in
        if (clear(x, y, s_env["obs"].player, s_env["map"]) and
                not hostile_ship_near(x, y, s_env["obs"].player, s_env["map"], ship["cargo"])):
            # If the current cell has more than the biggest amount of halite
            if s_env["map"][x][y]["halite"] > most_halite:
                most_halite = s_env["map"][x][y]["halite"]
                direction = directions_list[d]["direction"]
                direction_x = x
                direction_y = y
    # If the cell is safe to move in and has a substantial amount of halite
    if most_halite > s_env["low_amount_of_halite"] and direction != None:
        ships_data[ship["key"]]["direction_to_move"] = direction
        return True
    # If the current cell has the biggest amount of halite
    elif most_halite == s_env["map"][ship["x"]][ship["y"]]["halite"]:
        ships_data[ship["key"]]["direction_to_move"] = "HERE"
        return True
    return ""

def standard_patrol(s_env, ship):
    """ 
        the ship will move in expanding circles clockwise or counterclockwise
        until reaching maximum radius, then the radius will be minimal again
    """
    directions = ships_data[ship["key"]]["directions"]
    # set index of direction
    i = ships_data[ship["key"]]["directions_index"]
    direction_found = False
    for j in range(len(directions)):
        x = directions[i]["x"](ship["x"])
        y = directions[i]["y"](ship["y"])
        # If the cell is ok to move in
        if (clear(x, y, s_env["obs"].player, s_env["map"]) and
                (s_env["map"][x][y]["shipyard"] == s_env["obs"].player or
                not hostile_ship_near(x, y, s_env["obs"].player, s_env["map"], ship["cargo"]))):
            ships_data[ship["key"]]["moves_done"] += 1
            # if it was the last move in this direction
            if ships_data[ship["key"]]["moves_done"] >= ships_data[ship["key"]]["ship_max_moves"]:
                ships_data[ship["key"]]["moves_done"] = 0
                ships_data[ship["key"]]["directions_index"] += 1
                # If it is the last direction in a list
                if ships_data[ship["key"]]["directions_index"] >= len(directions):
                    ships_data[ship["key"]]["directions_index"] = 0
                    ships_data[ship["key"]]["ship_max_moves"] += 1
                    # If ship_max_moves reached maximum radius expansion
                    if ships_data[ship["key"]]["ship_max_moves"] > max_moves_amount:
                        ships_data[ship["key"]]["ship_max_moves"] = 1
            ships_data[ship["key"]]["direction_to_move"] = directions[i]["direction"]
            direction_found = True
            break
        else:
            # Loop through directions
            i += 1
            if i >= len(directions):
                i = 0
    # If the ship is not in the shipyard and a hostile ship is near
    if (not direction_found and s_env["map"][ship["x"]][ship["y"]]["shipyard"] == None and
            hostile_ship_near(ship["x"], ship["y"], s_env["obs"].player, s_env["map"], ship["cargo"])):
        # If there is enough halite to convert
        if ship["cargo"] >= conf.convertCost:
            return "conv"
        else:
            for i in range(len(directions)):
                x = directions[i]["x"](ship["x"])
                y = directions[i]["y"](ship["y"])
                # If it is the opponent's shipyard
                if s_env["map"][x][y]["shipyard"] != None:
                    ships_data[ship["key"]]["direction_to_move"] = directions[i]["direction"]
                    return True
            for i in range(len(directions)):
                x = directions[i]["x"](ship["x"])
                y = directions[i]["y"](ship["y"])
                # If it is an empty place
                if s_env["map"][x][y]["ship"] == None:
                    ships_data[ship["key"]]["direction_to_move"] = directions[i]["direction"]
                    return True
    return True

def get_directions(i0, i1, i2, i3):
    """ get a list of directions in a certain sequence """
    return [directions_list[i0], directions_list[i1], directions_list[i2], directions_list[i3]]

def hostile_ship_near(x, y, player, m, cargo):
    """ check if hostile ship is in one move away from game_map[x][y] and has less or equal halite """
    # m = game map
    n = get_c(y - 1)
    e = get_c(x + 1)
    s = get_c(y + 1)
    w = get_c(x - 1)
    if (
            (m[x][n]["ship"] != player and m[x][n]["ship"] != None and m[x][n]["ship_cargo"] <= cargo) or
            (m[x][s]["ship"] != player and m[x][s]["ship"] != None and m[x][s]["ship_cargo"] <= cargo) or
            (m[e][y]["ship"] != player and m[e][y]["ship"] != None and m[e][y]["ship_cargo"] <= cargo) or
            (m[w][y]["ship"] != player and m[w][y]["ship"] != None and m[w][y]["ship_cargo"] <= cargo)
        ):
        return True
    return False

def spawn_ship(actions, s_env, shipyard):
    """ spawn ship """
    s_env["swarm_halite"] -= conf.spawnCost
    actions[shipyard["key"]] = "SPAWN"
    s_env["map"][shipyard["x"]][shipyard["y"]]["ship"] = s_env["obs"].player
    s_env["ships_amount"] += 1
    return actions, s_env

def sd_swarm_halite_amount(s_env, shipyard):
    """ evaluate the amount of Swarm's halite """
    if s_env["swarm_halite"] >= conf.spawnCost:
        return ">=snc"
    return ""

def sd_ships_amount(s_env, shipyard):
    """ evaluate the number of ships in the Swarm """
    if s_env["ships_amount"] < s_env["ships_max_amount"]:
        return "<spma"
    return ""

def shipyard_clear(s_env, shipyard):
    """ check if this shipyard is clear """
    if clear(shipyard["x"], shipyard["y"], s_env["obs"].player, s_env["map"]):
        return True
    return ""

def record_found(s_env, unit, patterns):
    """ check if every pattern's result matches data returned from that pattern's function """
    for pattern in patterns:
        if pattern["fun"](s_env, unit) != pattern["result"]:
            return False
    return True

def this_is_new_ship(s_env, i):
    """ add this ship to ships_data """
    global movement_tactics_index
    ships_data[s_env["ships_keys"][i]] = {
        "moves_done": 0,
        "ship_max_moves": 1,
        "directions": movement_tactics[movement_tactics_index]["directions"],
        "directions_index": 0,
        "direction_to_move": "HERE"
    }
    movement_tactics_index += 1
    if movement_tactics_index >= movement_tactics_amount:
        movement_tactics_index = 0
    return s_env

def this_is_last_step(s_env, ship):
    """ check if it is the last step """
    if s_env["obs"].step == (conf.episodeSteps - 2) and ship["cargo"] >= conf.convertCost:
        return True
    return ""

def convert_ship(actions, s_env, ship):
    """ convert the ship into shipyard """
    actions[ship["key"]] = "CONVERT"
    s_env["map"][ship["x"]][ship["y"]]["ship"] = None
    s_env["shipyards_amount"] += 1
    s_env["swarm_halite"] = s_env["swarm_halite"] + ship["cargo"] - conf.convertCost
    return actions, s_env

def no_shipyards(s_env, ship):
    """ check if there is no shipyard and conversion of the ship is possible """
    shipyards_amount = len(s_env["shipyards_keys"])
    if (s_env["shipyards_amount"] == 0 and
            not hostile_ship_near(ship["x"], ship["y"], s_env["obs"].player, s_env["map"], ship["cargo"]) and
            (s_env["swarm_halite"] + ship["cargo"]) >= convert_threshold):
        return True
    return ""

def to_spawn_or_not_to_spawn(s_env):
    """ to spawn, or not to spawn, that is the question """
    # get ships_max_amount to decide whether to spawn new ships or not
    ships_max_amount = 9
    # If ships_max_amount is less than the minimal allowed amount of ships in the Swarm
    if ships_max_amount < ships_min_amount or s_env["obs"].step >= spawn_stop_step:
        ships_max_amount = ships_min_amount
    return ships_max_amount

def define_some_globals(configuration):
    """ define some of the global variables """
    global conf
    global convert_threshold
    global max_moves_amount
    global spawn_stop_step
    global globals_not_defined
    conf = configuration
    convert_threshold = conf.convertCost + conf.spawnCost * 2
    max_moves_amount = 3
    spawn_stop_step = conf.episodeSteps - 150
    globals_not_defined = False

def adapt_environment(observation, configuration):
    """ adapt the environment for the Swarm """
    s_env = {}
    s_env["obs"] = observation
    if globals_not_defined:
        define_some_globals(configuration)
    s_env["map"], s_env["average_halite"] = get_map_and_average_halite(s_env["obs"])
    s_env["low_amount_of_halite"] = s_env["average_halite"] * 0.75
    s_env["swarm_halite"] = s_env["obs"].players[s_env["obs"].player][0]
    s_env["swarm_shipyards_coords"], s_env["swarm_ships_coords"] = get_swarm_units_coords_and_update_map(s_env)
    s_env["ships_keys"] = list(s_env["obs"].players[s_env["obs"].player][2].keys())
    s_env["ships_values"] = list(s_env["obs"].players[s_env["obs"].player][2].values())
    s_env["shipyards_keys"] = list(s_env["obs"].players[s_env["obs"].player][1].keys())
    s_env["ships_max_amount"] = to_spawn_or_not_to_spawn(s_env)
    s_env["ships_amount"] = len(s_env["ships_keys"])
    s_env["shipyards_amount"] = len(s_env["shipyards_keys"])
    return s_env
    
def actions_of_ships(s_env):
    """ actions of every ship of the Swarm """
    actions = {}
    for i in range(len(s_env["ships_keys"])):
        # If this is a new ship
        if s_env["ships_keys"][i] not in ships_data:
            s_env = this_is_new_ship(s_env, i)
        ship = {
            "x": s_env["swarm_ships_coords"][i][0],
            "y": s_env["swarm_ships_coords"][i][1],
            "key": s_env["ships_keys"][i],
            "cargo": s_env["ships_values"][i][1]
        }
        # find the first suitable record in memory
        for record in memory["ships"]:
            if record_found(s_env, ship, record["patterns"]):
                # perform the action of suitable record
                actions, s_env = record["action"](actions, s_env, ship)
                break
    return actions
     
def actions_of_shipyards(actions, s_env):
    """ actions of every shipyard of the Swarm """
    # spawn ships from every shipyard, if possible
    for i in range(len(s_env["shipyards_keys"])):
        shipyard = {
            "x": s_env["swarm_shipyards_coords"][i][0],
            "y": s_env["swarm_shipyards_coords"][i][1],
            "key": s_env["shipyards_keys"][i]
        }
        # find the first suitable record in memory
        for record in memory["shipyards"]:
            if record_found(s_env, shipyard, record["patterns"]):
                # perform the action of suitable record
                actions, s_env = record["action"](actions, s_env, shipyard)
                break
    return actions


#GLOBAL_VARIABLES#############################################
conf = None
# max amount of moves in one direction before turning
max_moves_amount = None
# threshold of harvested by ship halite to convert
convert_threshold = None
# no ship spawning above ships_min_amount after this step
spawn_stop_step = None
# object with ship IDs and their data
ships_data = {}
# initial movement_tactics index
movement_tactics_index = 0
# minimum amount of ships that should be in the Swarm at any time
ships_min_amount = 3
# not all global variables are defined
globals_not_defined = True

# list of directions
directions_list = [
    {
        "direction": "NORTH",
        "x": lambda z: z,
        "y": lambda z: get_c(z - 1)
    },
    {
        "direction": "EAST",
        "x": lambda z: get_c(z + 1),
        "y": lambda z: z
    },
    {
        "direction": "SOUTH",
        "x": lambda z: z,
        "y": lambda z: get_c(z + 1)
    },
    {
        "direction": "WEST",
        "x": lambda z: get_c(z - 1),
        "y": lambda z: z
    }
]

# List of movement tactics
movement_tactics = [
    # N -> E -> S -> W
    {"directions": get_directions(0, 1, 2, 3)},
    # S -> E -> N -> W
    {"directions": get_directions(2, 1, 0, 3)},
    # N -> W -> S -> E
    {"directions": get_directions(0, 3, 2, 1)},
    # S -> W -> N -> E
    {"directions": get_directions(2, 3, 0, 1)},
    # E -> N -> W -> S
    {"directions": get_directions(1, 0, 3, 2)},
    # W -> S -> E -> N
    {"directions": get_directions(3, 2, 1, 0)},
    # E -> S -> W -> N
    {"directions": get_directions(1, 2, 3, 0)},
    # W -> N -> E -> S
    {"directions": get_directions(3, 0, 1, 2)},
]
movement_tactics_amount = len(movement_tactics)

# List of ship functions and lists of possible results
sp_funs_and_results = [
    {
        "key": "this_is_last_step",
        "fun": this_is_last_step,
        "results": [
            True
        ]
    },
    {
        "key": "no_shipyards",
        "fun": no_shipyards,
        "results": [
            True
        ]
    },
    {
        "key": "go_for_halite",
        "fun": go_for_halite,
        "results": [
            True
        ]
    },
    {
        "key": "standard_patrol",
        "fun": standard_patrol,
        "results": [
            True,
            # conv -> CONVERT
            "conv"
        ]
    }
]

# List of possible actions of ships
sp_actions = [
    {
        "key": "convert_ship",
        "action": convert_ship
    },
    {
        "key": "move_ship",
        "action": move_ship
    }
]

# List of shipyard functions and lists of possible results
sd_funs_and_results = [
    {
        "key": "sd_swarm_halite_amount",
        "fun": sd_swarm_halite_amount,
        "results": [
            # snc -> spawn cost
            ">=snc"
        ]
    },
    {
        "key": "sd_ships_amount",
        "fun": sd_ships_amount,
        "results": [
            # spma -> ships max amount
            "<spma"
        ]
    },
    {
        "key": "shipyard_clear",
        "fun": shipyard_clear,
        "results": [
            True
        ]
    }
]

# List of possible actions of shipyards
sd_actions = [
    {
        "key": "spawn_ship",
        "action": spawn_ship
    }
]

# memory of units
memory = {
    # ships list of memory records
    "ships": [
        {
            "patterns": [
                {
                    "key": sp_funs_and_results[0]["key"],
                    "fun": sp_funs_and_results[0]["fun"],
                    "result": sp_funs_and_results[0]["results"][0]
                }
            ],
            "action_key": sp_actions[0]["key"],
            "action": sp_actions[0]["action"]
        },
        {
            "patterns": [
                {
                    "key": sp_funs_and_results[1]["key"],
                    "fun": sp_funs_and_results[1]["fun"],
                    "result": sp_funs_and_results[1]["results"][0]
                }
            ],
            "action_key": sp_actions[0]["key"],
            "action": sp_actions[0]["action"]
        },
        {
            "patterns": [
                {
                    "key": sp_funs_and_results[3]["key"],
                    "fun": sp_funs_and_results[3]["fun"],
                    "result": sp_funs_and_results[3]["results"][1]
                }
            ],
            "action_key": sp_actions[0]["key"],
            "action": sp_actions[0]["action"]
        },
        {
            "patterns": [
                {
                    "key": sp_funs_and_results[2]["key"],
                    "fun": sp_funs_and_results[2]["fun"],
                    "result": sp_funs_and_results[2]["results"][0]
                }
            ],
            "action_key": sp_actions[1]["key"],
            "action": sp_actions[1]["action"]
        },
        {
            "patterns": [
                {
                    "key": sp_funs_and_results[3]["key"],
                    "fun": sp_funs_and_results[3]["fun"],
                    "result": sp_funs_and_results[3]["results"][0]
                }
            ],
            "action_key": sp_actions[1]["key"],
            "action": sp_actions[1]["action"]
        }
    ],
    # Shipyards list of memory records
    "shipyards": [
        {
            "patterns": [
                {
                    "key": sd_funs_and_results[0]["key"],
                    "fun": sd_funs_and_results[0]["fun"],
                    "result": sd_funs_and_results[0]["results"][0]
                },
                {
                    "key": sd_funs_and_results[1]["key"],
                    "fun": sd_funs_and_results[1]["fun"],
                    "result": sd_funs_and_results[1]["results"][0]
                },
                {
                    "key": sd_funs_and_results[2]["key"],
                    "fun": sd_funs_and_results[2]["fun"],
                    "result": sd_funs_and_results[2]["results"][0]
                }
            ],
            "action_key": sd_actions[0]["key"],
            "action": sd_actions[0]["action"]
        }
    ]
}

Swarm

 
#THE_SWARM####################################################
def swarm_agent(observation, configuration):
    """ RELEASE THE SWARM!!! """
    s_env = adapt_environment(observation, configuration)
    actions = actions_of_ships(s_env)
    actions = actions_of_shipyards(actions, s_env)
    return actions

调试智能体

现在我们将执行游戏循环，让你的智能体与随机智能体对战，直到游戏结束。

 
if "swarm_agent" in globals():
    # reset variables
    ships_data = {}
    movement_tactics_index = 0

    # Play as first position against a random agent.
    trainer = env.train([None, "random"])

    observation = trainer.reset()

    while not env.done:
        my_action = swarm_agent(observation, env.configuration)
        print("Step: {0}, My Action: {1}".format(observation.step, my_action))
        observation, reward, done, info = trainer.step(my_action)
        # env.render(mode="ipython", width=100, height=90, header=False, controls=False)
    env.render()

我们将开发一些函数，它们共同构成了遗传算法的基本组成部分，旨在在哈利特环境中进化你的智能体的记忆。该算法的目标是识别在系列回合中产生最佳哈利特收集结果的记忆配置。

 
import random
import copy

def get_average_halite(results):
    """ get average halite of agent from results of evaluated episodes """
    halite_sum = 0
    for r in results:
        halite_sum += r[0]
    return halite_sum / len(results)

def set_possible_records_and_patterns_amounts():
    """ set maximum possible amounts of memory records and patterns in those records """
    global possible_records_amount
    global possible_patterns_amount
    possible_records_amount = len(ea_actions) * len(ea_funs_and_results)
    possible_patterns_amount = len(ea_funs_and_results)
    for fun in ea_funs_and_results:
        possible_records_amount *= len(fun["results"])
        possible_patterns_amount *= len(fun["results"])
    possible_records_amount -= 1
    possible_patterns_amount -= 1

def mutations():
    """ mutate each specimen of the population """
    for the specimen in population:
        # remove some records from specimen
        records_to_remove = random.randint(0, len(specimen) - 1)
        for i in range(records_to_remove):
            specimen.pop(random.randint(0, len(specimen) - 1))
        # clone some records within specimen
        records_to_clone = random.randint(0, len(specimen))
        for i in range(records_to_clone):
            specimen.append(copy.deepcopy(specimen[random.randint(0, len(specimen) - 1)]))
        # mutate some actions in existing records
        actions_to_mutate = random.randint(0, len(specimen))
        for i in range(actions_to_mutate):
            record_index = random.randint(0, len(specimen) - 1)
            new_action_index = random.randint(0, len(ea_actions) - 1)
            specimen[record_index]["action_key"] = ea_actions[new_action_index]["key"]
            specimen[record_index]["action"] = ea_actions[new_action_index]["action"]
        # mutate some lists of patterns in existing records
        patterns_lists_to_mutate = random.randint(0, len(specimen) - 1)
        for i in range(patterns_lists_to_mutate):
            # remove some patterns from a record's list of patterns
            patterns_to_remove = random.randint(0, len(specimen[i]["patterns"]) - 1)
            for j in range(patterns_to_remove):
                specimen[i]["patterns"].pop(random.randint(0, len(specimen[i]["patterns"]) - 1))
            # clone some patterns within record
            patterns_to_clone = random.randint(0, len(specimen[i]["patterns"]))
            for j in range(patterns_to_clone):
                specimen[i]["patterns"].append(
                    copy.deepcopy(specimen[i]["patterns"][random.randint(0, len(specimen[i]["patterns"]) - 1)]))
            # mutate some patterns in a record's list of patterns
            patterns_to_mutate = random.randint(0, len(specimen[i]["patterns"]))
            for j in range(patterns_to_mutate):
                # p_i -> pattern index
                p_i = random.randint(0, len(specimen[i]["patterns"]) - 1)
                # n_p_i -> new pattern index
                n_p_i = random.randint(0, len(ea_funs_and_results) - 1)
                specimen[i]["patterns"][p_i]["key"] = ea_funs_and_results[n_p_i]["key"]
                specimen[i]["patterns"][p_i]["fun"] = ea_funs_and_results[n_p_i]["fun"]
                results_index = random.randint(0, len(ea_funs_and_results[n_p_i]["results"]) - 1)
                specimen[i]["patterns"][p_i]["result"] = ea_funs_and_results[n_p_i]["results"][results_index]

def fitness(memory_key):
    """ evaluate each specimen's fitness """
    global best_specimen
    global best_specimen_fitness
    for the specimen in population:
        # replace the memory of the unit with the specimen
        memory[memory_key] = specimen
        specimen_fitness = get_average_halite(evaluate("halite", [swarm_agent],
                                num_episodes=episodes, configuration={"agentExec": "LOCAL"}
                           ))
        if specimen_fitness > best_specimen_fitness:
            best_specimen_fitness = specimen_fitness
            best_specimen = copy.deepcopy(specimen)
    
def selection():
    """ select specimens for next generation """
    global population
    population = []
    for i in range(population_size):
        population.append(copy.deepcopy(best_specimen))

在这里，我们的目标是使用进化策略迭代地增强船只的记忆配置，直到我们找到一种能够最大限度地提高船只在哈利特环境中性能的配置。

 
# amount of generations
generations_amount = 10
# size of the population
population_size = 5
# list of specimens
population = []
# maximum possible amount of memory records
possible_records_amount = 0
# maximum possible amount of patterns in a record
possible_patterns_amount = 0
# number of episodes for fitness evaluation
episodes = 1
# List of possible actions
ea_actions = sp_actions
# List of functions and lists of possible results
ea_funs_and_results = sp_funs_and_results
# best specimen
best_specimen = copy.deepcopy(memory["ships"])
# best specimen fitness
best_specimen_fitness = get_average_halite(evaluate("halite", [swarm_agent],
                            num_episodes=episodes, configuration={"agentExec": "LOCAL"}
                        ))

# create an initial population
selection()

# evolve memory of ships with an evolutionary algorithm
for i in range(generations_amount):
    mutations()
    fitness("ships")
    selection()

# replace memory of ships with best specimen
memory["ships"] = copy.deepcopy(best_specimen)
print("Best specimen fitness: {0}, best specimen: {1}".format(best_specimen_fitness, best_specimen))

此过程的目标是使用进化方法迭代地增强船坞的记忆配置，直到找到一种能够最大限度地提高船坞在哈利特环境中性能的配置。

 
# amount of generations
generations_amount = 10
# size of the population
population_size = 5
# list of specimens
population = []
# maximum possible amount of memory records
possible_records_amount = 0
# maximum possible amount of patterns in a record
possible_patterns_amount = 0
# number of episodes for fitness evaluation
episodes = 1
# List of possible actions
ea_actions = sd_actions
# List of functions and lists of possible results
ea_funs_and_results = sd_funs_and_results
# best specimen
best_specimen = copy.deepcopy(memory["shipyards"])
# best specimen fitness
best_specimen_fitness = get_average_halite(evaluate("halite", [swarm_agent],
                            num_episodes=episodes, configuration={"agentExec": "LOCAL"}
                        ))

# create an initial population
selection()

# evolve memory of shipyards with an evolutionary algorithm
for i in range(generations_amount):
    mutations()
    fitness("shipyards")
    selection()

# replace memory of shipyards with best specimen
memory["shipyards"] = copy.deepcopy(best_specimen)
print("Best specimen fitness: {0}, best specimen: {1}".format(best_specimen_fitness, best_specimen))

评估智能体

现在我们将评估智能体。

 
def mean_reward(rewards):
    wins = 0
    ties = 0
    loses = 0
    for r in rewards:
        r0 = 0 if r[0] is None else r[0]
        r1 = 0 if r[1] is None else r[1]
        if r0 > r1:
            wins += 1
        elif r1 > r0:
            loses += 1
        else:
            ties += 1
    return f'wins={wins/len(rewards)}, ties={ties/len(rewards)}, loses={loses/len(rewards)}'

# Run multiple episodes to estimate its performance.
# Setup agentExec as LOCAL to run in memory (runs faster) without process isolation.
print("Swarm Agent vs Random Agent:", mean_reward(evaluate(
    "halite",
    [swarm_agent, "random", "random", "random"],
    num_episodes=10, configuration={"agentExec": "LOCAL"}
)))

测试智能体

现在我们将测试智能体。

 
env.run([swarm_agent])
env.render(mode="ipython", width=800, height=600)

输出

下一主题机器学习中的 L1 和 L2 正则化方法

群体智能

群体智能原理

创建哈利特环境

Swarm

调试智能体

评估智能体

测试智能体

联系信息

关注我们

教程

面试题

在线编译器

Python

Java

.Net Framework

AI, ML and Data Science

Cloud Technology

B.Tech and MCA

Web Technology

PHP

Software Testing

Technical Interview

Java Interview

Python

Web Interview

Database Interview

B.Tech / MCA

Important Interview

Software Testing Interview

Company Interviews

Online Compilers

Multiple Choice Questions

机器学习

监督式学习

分类

杂项

相关教程

面试题

群体智能

群体智能原理

创建哈利特环境

Swarm

调试智能体

评估智能体

测试智能体

相关帖子

机器学习中的转置卷积

衡量模型不确定性的方法

使用 Keras 进行 YOLOv3 目标检测

机器学习中的电力消耗预测

深度学习中缩放数据、模型大小和性能

EfficientNet：机器学习模型架构的突破

机器学习中的过拟合

机器学习中的嵌入是什么？

神经网络中 Batch 和 Epoch 的区别

广义估计方程简介

订阅 Tpoint Tech

联系信息

关注我们

教程

面试题

在线编译器