How to add fieldset in Magento custom admin form when field select value changed

For that you can use field dependencies feature of Magento, like here is the example

$status = $fieldSet->addField('status', 'select', array(
    'label'     => Mage::helper('module')->__('Status'),
    'name'      => 'status',
    'values'    => array(
        'Approved' => 'Approved',
        'Denied'   => 'Denied'
    )
));

$denyReason = $fieldSet->addField('denial_reason', 'textarea', array(
    'label'     => Mage::helper('module')->__('Denial Reason'),
    'name'      => 'denial_reason'
));

And add your dependencies

$this->setForm($form);
$this->setChild('form_after', $this->getLayout()->createBlock('adminhtml/widget_form_element_dependence')
    ->addFieldMap($status->getHtmlId(), $status->getName())
    ->addFieldMap($denyReason->getHtmlId(), $denyReason->getName())                     
    ->addFieldDependence(
        $denyReason->getName(),
        $status->getName(),
        'Denied'
    )
);

$denyReason field only display If status is set to Denied. According to this you can define your field dependencies.

use setAfterElementHtml function. Suppose, you want to show/hide the following field:

$fieldset->addField('field_to_hide', text, array(
    'label'  => $this->__('Test'),
));

Here is the dropdown/select field. It’s a yes/no field. Suppose, upon selecting ‘Show’, you will need to show the above mentioned field. And, upon selecting ‘Hide’, you will need to hide the above field.

Note the onChange parameter here. It’s calling showHideField() JS function.

$selectField = $fieldset->addField('select_field', 'select', array(
    'label' => 'Show/Hide Selection',
    'name' => 'select_field',           
    'values' => array(
        array(
            'value' => 1,
            'label' => 'Show',
        ),
        array(
            'value' => 0,
            'label' => 'Hide',
        ),
    ),
    'onChange'  => 'showHideField()',
));

The setAfterElementHtml is called which contains Javascript code to show/hide field

$selectField->setAfterElementHtml('
    <script>
    function showHideField() {                                              
        $("field_to_hide").toggle()
    }
    </script>
');