Dynamic menus & the menu API

Most of the time a menu is a static object that doesn't grow or shrink in response to user input. There are times, however, when you need a menu that will dynamically change during run time.

I was faced with this requirement while developing on-line trading tools for an Option Trading company. The program in question was designed to display grids of information that could be tailored by the Trader. As the Trader created, renamed and deleted these views, this structure needed to be represented in the menu. Optionally, the trader could drill further down each view to a single Option Symbol, which was represented as a sub-menu item.

This article will show how to create a dynamic menu that will grow or shrink during run-time as needed and respond to the user.

First things first – Sub-classing the Form

Although the Menu APIs will allow us to create menus and sub-menus, they won't do us much good by themselves. Menus created at run-time are not known to Visual Basic, which only knows about menus created by the menu editor. To be able to respond to menu clicks on our dynamically created menus, we will have to trap all messages sent to the form. If the message is a menu click, we can determine which menu item was selected. This process of trapping messages being sent to the form is called "sub-classing".

One note about sub-classing that should be mentioned. Be careful about saving your work whenever working with APIs in general, and specifically when sub-classing a form. You will find some peculiar behaviour when debugging. For example, if you set a breakpoint that breaks in the middle of processing a message, the keystrokes will be mostly ignored by the IDE. While you can use the function keys to single-step, you will be unable to use immediate mode or add variables to the watch window. Occasionally, your session may crash entirely and you will lose your work if not previously saved.

Sub-classing the form is done with the use of two APIs: SetWindowLong and CallWindowProc. The call to SetWindowLong uses the AddressOf operator to pass the address of our message handling function, SubClassHandler. This is done in the Form_Load event as follows:

g_lpPrevWndFunc = SetWindowLong( _
	frmForm.hWnd, _
	GWL_WNDPROC, _
	AddressOf SubClassHandler)

The return value is a pointer to the system handler and is saved in the global variable g_lpPrevWndFunc for use later. You should end sub-classing in the Form_Unload event as follows:

lngRC = SetWindowLong(frmForm.hWnd, _
	GWL_WNDPROC, _
	g_lpPrevWndFunc)

The parameter to the AddressOf method must be a function that accepts four Long Integers by Reference and returns a Long Integer:

Public Function SubClassHandler( _
	ByVal hWnd As Long, _
	ByVal lngMsg As Long, _
	ByVal wParam As Long, _
	ByVal lParam As Long) As Long

This routine, which should be declared in a module, will receive every message sent to the form from Windows. For a menu click, lngMsg will be set to a value of WM_COMMAND, lParam to a value of zero and wParam to the menu ID of the menu item. The menu processing routine is called whenever Windows sends us the appropriate message:

If lngMsg = WM_COMMAND Then
	If lParam = 0 Then
		'Menu click here, pass the form
		'handle and menu ID
		Call ProcessMenu(hWnd, wParam)
	End If
End If

The last line of code in the message handing routine (SubClassHandler) must return control to Windows with the following statement:

SubClassHandler = CallWindowProc( _
	g_lpPrevWndFunc, _
	hWnd, lngMsg, wParam, lParam)

Adding Menus to Visual Basic Forms

Now with the issue of sub-classing behind us, let's look at the mechanics of building the menus. One limitation we run into with Visual Basic 6.0 in regard to menus is that we can not actually append a menu to a form that did not already have a menu to begin with at design time. While this may initially appear to defeat the requirements, this problem actually has a very easy work-around. We will create one sub-menu/menu item pair at design time, which are initially left invisible. During execution, we will set these to visible to add the first sub-menu to the menu bar, and use the Menu APIs to add additional sub-menus.

Menus, sub-menus, items and handles

First of all, a little terminology is in order. A menu as it relates to the Menu APIs is an object that attaches to a form and contains one or more menu items. A menu item may be a single entity or contain a sub-menu, which itself will normally have one or more additional menu items. Each menu item has a caption and a menu ID, which is also referred to as the menu command. The menu ID or command is a numeric identifier normally supplied by the programmer when the item is inserted or appended into a menu or sub-menu. The exception to this is a menu item that points to a sub-menu, which always has an ID of -1.

The term menu bar will be used in this article to describe the menu items that appear at the top of a form. Using the Visual Basic IDE as an example, the menu bar consists of thirteen items ranging from File to Help. Each of these menu items point to sub-menus with additional items. Some of these menu items also point to sub-menus – for example, selecting Format|Align reveals another sub-menu.

Every form, menu or sub-menu has a handle. A handle is a unique long integer assigned by Windows that is used by the system to identify a particular instance of an object. You do not assign handles directly, but rather will generally access a parameter or property or call an API to retrieve a handle, which will be passed to another API that requires a handle. While a form handle cannot be used in place of a menu handle and vice-versa, the only real difference between a menu handle and a sub-menu handle, as far as we will be concerned, is the API that you use to retrieve them (more on this below).

Other than that, they are used interchangeably with APIs that require a menu handle. The menu handle and menu ID together are used to identify which item on which sub-menu was clicked.

The user is presented with a dialog box when a menu item has been clicked

A look at the Menu APIs

At first glance, the menu APIs may appear a bit intimidating. There are quite a few of them, and as we will see later – at least one has a name which does not seem to match its function. For the purposes of this article we will only look at the APIs needed to create dynamic menus, which we will divide into two broad categories. The first category involves the actual process of creating and removing menu entries, and the second category serves to identify which menu item has been clicked by the user.

Adding and removing items from an existing menu is accomplished by using the cleverly named AppendMenu, InsertMenu and RemoveMenu APIs. RemoveMenu removes a menu item and AppendMenu adds an item to the end of the list. InsertMenu is similar to AppendMenu, but accepts an additional parameter that specifies the position at which to add the new menu item. An enhanced version of this functionality is provided by InsertMenuItem, which does the same thing but provides additional control over the look and feel.

To add a sub-menu, we use CreatePopupMenu along with AppendMenu, InsertMenu or InsertMenuItem to attach it to the parent menu and to add items to it. CreatePopupMenu's name is a bit misleading if you ask me. One tends to think of a popup menu as something that appears when the right-mouse button is clicked, but in reality a popup in this sense is also a sub-menu. This API is used to add a sub-menu to the menu bar or a sub-menu to a menu item. Whenever you call one of the above APIs, you should follow up with a call to DrawMenuBar, which will refresh the menu display on the form.

We identify menus by using the following APIs:

• GetMenu
• GetSubMenu
• GetMenuItemCount
• GetMenuItemID
• GetMenuString

GetMenu will give us the menu handle to the menu bar, which is the basis for finding our way through the rest of the structure. GetSubMenu will similarly return the menu handle of a sub-menu given the menu handle and position of its parent. The GetMenuItemCount API accepts a menu handle and returns the number of items in the corresponding menu or sub-menu. This count only refers to the items at the same level as the menu handle that was passed. You must recursively call this routine for any sub-menus to get the entire count of all possible menu items. The next two APIs, GetMenuString and GetMenuItemID are used to identify a specific menu item. GetMenuString returns the caption from a menu item based on its menu handle and either its position or its menu ID. GetMenuItemID returns the menu ID based on a menu handle and position. Remember that the menu ID will be -1 if it is pointing to a sub-menu, so a call to GetSubMenu will be needed to further iterate a menu item with a menu ID of -1.

Putting it all to use

Now that we have all the players in the game, let's see how to put the pieces together to do some real work. As stated above, the first sub-menu and its first menu item are added at design time with their visible attributes set to FALSE. These attributes are set to TRUE to add the first sub-menu to the menu bar. Additional items are added to the menu bar by calling the Visual Basic routine, AddPopupMenu, described below.

If Not mnuSubMenu.Visible Then
	mnuSubMenu.Visible = True
	mnuSubMenu.Caption = strSubMenuCaption
	mnuMenuItem.Caption = strMenuItemCaption
Else
	'Use the APIs to create and add
	'a new sub-menu
	AddPopupMenu GetMenu(Me.hWnd), _
		strSubMenuCaption, _
		strMenuItemCaption
End If

AddPopupMenu is passed a menu handle, which will accept the new sub-menu, along with the new menu/sub-menu captions. This routine is used both to add a new sub-menu to the menu bar and to add a new sub-menu to an existing sub-menu.

AddPopupMenu first creates a new menu handle by a call to the CreatePopupMenu API. The new menu item is added to the new sub-menu using AppendMenu. Next, the new sub-menu is appended to the parent menu using AppendMenu or inserted using InsertMenu depending on whether or not the optional parameter, varPosition, was passed:

Public Sub AddPopupMenu( _
	ByVal hMenu As Long, _
	strItemCaption As String, _
	strSubItemCaption As String, _
	Optional varPosition As Variant)

	Dim hPopupMenu As Long
	Dim lngRC As Long

	'Create a new popup menu handle
	hPopupMenu = CreatePopupMenu()

	'Append the new item to the 
	'new sub-menu
	lngRC = AppendMenu(hPopupMenu, _
		MF_STRING, _
		GetNextMenuNumber(), _
		strSubItemCaption)

	'Append the new sub-menu if no 
	'position passed, else insert
	'at varPosition
	If IsMissing(varPosition) Then
		lngRC = AppendMenu(hMenu, _
			MF_POPUP, hPopupMenu, _
			strItemCaption)
	Else
		lngRC = _
		InsertMenu(hMenu,varPosition, _
		MF_POPUP + MF_BYPOSITION, _
		hPopupMenu, strItemCaption)
	End If

End Sub

Note that the third parameter to AppendMenu will be either a unique menu identifier if MF_STRING is specified, or a menu handle if MF_POPUP is specified. Use MF_STRING to append a menu item and MF_POPUP to append a sub-menu. InsertMenu additionally wants a flag to indicate whether the second parameter refers to a position or menu ID. Multiple flags are specified by adding them together.

The GetNextMenuNumber function (not listed) returns a unique menu identifier, which is then incremented and stored in a static variable. The actual value returned doesn't matter as long as it is unique. The menu item added during design time will have a MenuID of 2, so the first number returned must be at least 3, but I like to start with 100 just to be robust.

Deleting an item can be done using the RemoveMenu API, but this will leave an orphaned parent whenever the last menu item is deleted. A better solution is to recursively delete any parents when the last menu item is removed:

Public Sub DeleteMenuItem( _
	ByVal hMenuBar As Long, _
	hDeleteMenu As Long, _
	lngDeletePosition As Long)
	Dim lngItemCount As Long
	Dim hParentMenu As Long
	Dim lngParentPosition As Long
	Dim bDeleteParent As Boolean
	Dim lngRC As Long

	'If the item count is 1, this is 
	'the last menu item and we want
	'to also delete the parent.
	If GetMenuItemCount(hDeleteMenu) _
		= 1 Then
		'We want to delete the parent
		'here, grab the parent's menu 
		'handle and position.
		'We should always get a TRUE
		'here...
		bDeleteParent = _
			GetParentMenu(hMenuBar, _
			hDeleteMenu, hParentMenu, _
			lngParentPosition)
	Else
		bDeleteParent = False
	End If

	'Delete this item and recurse to
	'delete the parent if applicable.
	lngRC = RemoveMenu(hDeleteMenu, _
		lngDeletePosition, _
		MF_BYPOSITION)
	If bDeleteParent Then
		DeleteMenuItem hMenuBar, _
			hParentMenu,_
			lngParentPosition
	End If

End Sub

Since there is no API to return the menu handle of a parent given its child, we will have to write our own routine:

Public Function GetParentMenu( _
	ByVal hMenuBar As Long, _
	ByVal hChildMenu As Long, _
	ByRef hParentMenu As Long, _
	ByRef hParentPosition As Long) _
	As Long

	Dim lngPosition As Long
	Dim lngCount As Long
	Dim lngMenuID As Long
	Dim hSubMenu As Long
	Const NO_PARENT = -1

	'Default to no parent
	GetParentMenu = NO_PARENT

	'Get the number of items at this
	'level
	lngCount = _
		GetMenuItemCount(hMenuBar)

	'Loop for each item
	For lngPosition = 0 To lngCount - 1

		'Check each sub-menu looking 
		'for hChildMenu
		lngMenuID = _
			GetMenuItemID(hMenuBar,_
			lngPosition)
		If lngMenuID = -1 Then
			'We have a sub-menu here.
			'We are done if the sub-menu
			'handle matches...
			hSubMenu = _
				GetSubMenu(hMenuBar, _
				lngPosition)
			If hSubMenu = hChildMenu _
				Then
				hParentMenu = hMenuBar
				hParentPosition = _
					lngPosition
				GetParentMenu = True
			Else
				'Didn't match here,
				'recurse back to check 
				'this sub-menu
				GetParentMenu = _
					GetParentMenu( _
					hSubMenu, _
					hChildMenu, _
					hParentMenu, _
					hParentPosition)
			End If
		End If

	Next lngPosition

End Function

We now have all we need to add sub-menus to the menu bar, items to the sub-menus and sub-menus to the sub-menus.

A dynamically created menu

None of this will be much good to us though, if we can't determine which item was clicked by the user. Luckily for us, the message sent by Windows contains the handle of the form that owns the menu along with the menu ID of the item that was clicked. We can use this information to get the caption, sub-menu handle and position of this menu item.

Here is a function that will accept the form handle and menu id and return the menu's caption:

Public Function _
	GetMenuCaptionByCommand( _
	ByVal hWnd As Long, _
	lngMenuCommand As Long) As String

	Dim lngRC As Long
	Dim lngMenuCount As Long
	Dim hMenu As Long
	Dim hSubMenu As Long
	Dim lngItem As Long
	Dim strString As String
	Dim lngMaxCount As Long
	Dim lngFlag As Long

	'Get the form's menu bar
	hMenu = GetMenu(hWnd)
	If hMenu <> 0 Then
		'Initialize the buffer
		strString = Space$(256)
		'lngRC gets the number of 
		'characters returned...
		lngRC = GetMenuString( _
			hMenu, lngMenuCommand, _
			strString, Len(strString), _
			MF_BYCOMMAND)
		'Return the item caption
		GetMenuCaptionByCommand = _
			Left$(strString, lngRC)
	Else
		'Something went wrong here –
		'nothing found.
		GetMenuCaptionByCommand = ""
	End If

End Function

Now we need the ability to convert a menu ID to a menu handle and position. These will be needed to append, insert or delete menu items and sub-menus. We will write a function that iterates through the menu tree structure, checking each menu ID found along the way for a match. This is a recursive process that begins at the menu bar. We will use GetMenuItemCount to get the number of items and loop for each item. Each menu ID will be compared until the ID we are searching for is encountered or the last item has been checked. The recursion comes in whenever GetMenuItemID returns a value of -1, which indicates this item points to a sub-menu. In this case, we obtain the sub-menu handle using the GetSubmenu API and repeat the process, passing the sub-menu handle back recursively. The following function, FindMenuByID demonstrates this process:

Public Function FindMenuByID( _
	ByVal hMenu As Long, _
	ByVal lngFindMenuID As Long, _
	ByRef hFoundMenu As Long, _
	ByRef lngFoundPosition As Long) _
	As Boolean

	Dim lngPosition As Long
	Dim lngCount As Long
	Dim lngMenuID As Long
	Dim hSubMenu As Long

	FindMenuByID = False

	'Get the number of items at
	'this level
	lngCount = GetMenuItemCount(hMenu)

	'Loop for each item
	For lngPosition = 0 To lngCount - 1

		'Get the menu ID for the item
		'at this position
		lngMenuID = _
			GetMenuItemID(hMenu, _
			lngPosition)

		'We are done if tnis ID
		'matches lngFindMenuID
		If lngMenuID = lngFindMenuID _
			Then
			hFoundMenu = hMenu
			lngFoundPosition = _
				lngPosition
			FindMenuByID = True
			Exit Function
			'No match here – is this
			'a sub-menu?
		ElseIf lngMenuID = -1 Then
			'We have a sub-menu here get
			'the sub-menu handle.
			hSubMenu = _
				GetSubMenu(hMenu, _
				lngPosition)

			'Recurse back with the
			'sub-menu handle.
			'We are done if we got
			'a hit.
			If FindMenuByID(hSubMenu, _
				lngFindMenuID, _
				hFoundMenu, _
				lngFoundPosition) Then
				FindMenuByID = True
				Exit For
			End If
		End If
	Next lngPosition

End Function

Writing the message handler that will detect and process menu clicks is a relatively simple matter. Recall that this function must accept four long integers and return a long integer.

The first parameter contains the form handle of the form receiving the message. The second parameter is the type of message being sent to the form. A value of WM_COMMAND indicates a menu function. In the case of a WM_COMMAND message, the next parameter will contain the menu ID of the menu that was clicked and the forth parameter will be zero. If we detect a menu click, we will pass the form handle and the menu ID to a menu processing routine, which will react to the user's input:

Public Function SubClassHandler( _
	ByVal hWnd As Long, _
	ByVal lngMsg As Long, _
	ByVal wParam As Long, _
	ByVal lParam As Long) As Long

	Dim strMenuClickedMessage As String

	'Menu click?
	If lngMsg = WM_COMMAND Then
		If lParam = 0 Then
			'Menu click here, pass the
			'form handle and menu ID
			Call ProcessMenu(hWnd, _
				wParam)
		End If
	End If

	'Return control to Windows
	SubClassHandler = CallWindowProc( _
		g_lpPrevWndFunc, hWnd, _
		lngMsg, wParam, lParam)

End Function

Dynamic Menu – Program Example

Let's look at a sample menu processing routine that demonstrates everything we've discussed. This routine will be called by the sub-class handler when the user has clicked a menu item. We will show a modal dialog to display the caption of the item just clicked. From here, the user can choose to insert a menu item or sub-menu before or after this point, delete this item or continue with no further action.

The menu processing routine first uses our Visual Basic routine, GetMenuCaptionByCommand to get the caption of the menu item.

Next a public function, ProcessMenuClick defined in the form frmMenuItem (not listed), is called, which will display the caption, prompt the user and return the result.

If the user has selected an insert or delete, we call FindMenuByID, which should return TRUE and update hFoundMenu and lngFoundPosition with the sub-menu handle and position for this item. Now we call InsertMenu, AddPopupMenu or DeleteMenuItem as appropriate.

The functions GetMenuItemCaption and GetSubMenuCaptions (also not listed) prompt the user for either a menu item caption, when an item is being inserted, or for sub-menu and menu item captions when inserting a sub-menu. Each routine returns TRUE to continue or FALSE if the user has canceled.

Private Sub ProcessMenu(hWnd As Long, _
				lngMenuCommand As Long)

	Dim lngRC As Long
	Dim strSubMenuCaption As String
	Dim strMenuItemCaption As String
	Dim hMenu As Long
	Dim hSubMenu As Long
	Dim lngMenuID As Long
	Dim hParentMenu As Long
	Dim lngParentPosition As Long
	Dim hFoundMenu As Long
	Dim lngFoundPosition As Long
	Dim maAction As MenuAction
	Dim bRemoveParent As Boolean

	'Get the menu caption of the clicked item
	strMenuItemCaption = _
		GetMenuCaptionByCommand(hWnd, lngMenuCommand)

	'Display the menu caption just clicked and
	'prompt the user
	maAction = frmMenuItem.ProcessMenuClick(_
		strMenuItemCaption, "")

	'Continue or take some action?
	If maAction <> ACTION_CONTINUE Then
		hMenu = GetMenu(hWnd)
		'FindMenuByID will return TRUE if lngMenuCommand is found,
		' and update hFoundMenu and lngFoundPosition with the
		' sub-menu handle and position.
		If FindMenuByID(hMenu, lngMenuCommand, hFoundMenu, _
				lngFoundPosition) Then
			Select Case maAction
			'Insert menu item before
			Case ACTION_INSERT_ITEM_BEFORE
				'GetMenuItemCaption prompts the user for a
				'menu item caption, updates strMenuCaption and
				'returns TRUE unless user canceled.
				If GetMenuItemCaption(strMenuItemCaption) Then
					lngRC = InsertMenu(hFoundMenu, lngFoundPosition, _
					MF_STRING + MF_BYPOSITION, _
					GetNextMenuNumber(), strMenuItemCaption)
				End If

			'Insert menu item after
			Case ACTION_INSERT_ITEM_AFTER
				If GetMenuItemCaption(strMenuItemCaption) Then
					lngRC = InsertMenu(hFoundMenu, _
					lngFoundPosition + 1, MF_STRING _
					+ MF_BYPOSITION, GetNextMenuNumber, strMenuItemCaption)
				End If

			'Insert sub-menu before
			Case ACTION_INSERT_SUBMENU_BEFORE
				'GetSubMenuCaptions prompts the user for sub-menu,
				'menu item captions, updates strMenuCaption,
				'strSubMenuCaption and returns
				'TRUE unless user cancel
				If GetSubMenuCaptions(strSubMenuCaption, _
				strMenuItemCaption) Then
					AddPopupMenu hFoundMenu, strSubMenuCaption, _
										strMenuItemCaption, lngFoundPosition
				End If

			'Insert sub-menu after
			Case ACTION_INSERT_SUBMENU_AFTER
				If GetSubMenuCaptions(strSubMenuCaption, _
				strMenuItemCaption) Then
					AddPopupMenu hFoundMenu, strSubMenuCaption, _
					strMenuItemCaption, lngFoundPosition + 1
				End If

			'Delete menu item
			Case ACTION_DELETE
				'Delete menu will delete this item and recursively
				'delete any orphaned parents.
				DeleteMenuItem hMenu, hFoundMenu, lngFoundPosition

			End Select

			'Refresh the menu on the form
			lngRC = DrawMenuBar(hWnd)
		End If 'FindMenuByID(...
	End If 'maAction <> ACTION_CONTINUE

End Sub

The entire project can be downloaded from www.skycoder.com/downloads.